Making Friends With Command Line Perl

Making friends with command line tools usually pays off. There's other documentation to explain what the components do, but often tying things together requires a little experience, so here are some quick examples to learn from. Usually, you start with a simple task, and you quickly realize there's a special case to deal with, or you need to apply this to all the files in a directory, etc. In any case, these examples start simple and grow, just like command lines in real life usually do.

This is ZSH syntax.

Beginner's Example
Switching leading spaces to tabs in a source file:

perl -pwe '0 while s/^(\t*) {8}/$1\t/gs' < /tmp/old > /tmp/new

Maybe you'd like to see the diff between the old and new immediately:

perl -pwe '0 while s/^(\t*) {8}/$1\t/gs' < /tmp/old | tee /tmp/new | diff /tmp/old -

What if you have 20 files in a directory to do this for?

mkdir /tmp/new for file in *; do perl -pwe '0 while s/^(\t*) {8}/$1\t/gs' < ${file} > /tmp/new/${file}; done

Renaming Lots Of Files
Let's say you want to lowercase all the PNG file extensions because an annoying windows machine goofed them up. Remove the echo if you're ready to commit, but always do a test run first with the echo to be sure you haven't made a typo. It won't be long before you're glad you did.

unsetopt sh_word_split for oldname in *.PNG; do (echo $oldname | perl -pwe 's/\.PNG$/.png/gs' | read newname && echo mv -iv $oldname $newname) || break; done

To lowercase all file extensions:

unsetopt sh_word_split for oldname in *; do (echo $oldname | perl -pwe 's/\.([^\.]+)$/.\L$1/gs' | read newname && echo mv -iv $oldname $newname) || break; done

If $oldname == $newname, mv will ask you to overwrite. If this troubles you, it's easy to work around. If you're confident and lazy change -i to -f. Otherwise test x$oldname != x$newname. This starts to get a bit clumsy for beginners though:

unsetopt sh_word_split for oldname in *; do (echo $oldname | perl -pwe 's/\.([^\.]+)$/.\L$1/gs' | read newname && if test x$oldname != x$newname; then mv -fv $oldname $newname; fi) || break; done

CSV Files
Here's a quick way to deal with CSV files properly. Sure splitting on comma roughly works, but only when you're sure there aren't any commas in the fields.

Let's say you want to clear columns 9 and 10 when they duplicate columns 7 and 8. This separates each line ($_) into an array of fields (@_), then combines it back. The BEGIN block isn't entirely necessary in this example (you could instead do the setup code during each iteration), but demonstrates how to do some setup code before reading the first line. Note that you can replace the if statement with whatever transformation you're interested in.

perl -pwe 'BEGIN {use Text::CSV_XS; $csv = Text::CSV_XS->new; $csv->always_quote(1);} $csv->parse($_); @_ = $csv->fields; if($_[7] eq $_[9] and $_[8] eq $_[10]) {$_[9] = ""; $_[10] = "";} $csv->combine(@_); $_ = $csv->string. "\n"'