Antelope Love Fan

While you can live a (mostly) happy Linux existence with just GUI tools, I think a strong base working on the command-line can drastically improve your flexibility and efficiency. I urge everyone to spend some time studying all of the great Bash tutorials out there.

I am constantly learning new things. For instance, I had never considered combining multiple commands into a single alias before.

Which leads us to my favorite Bash trick of all time; the single thing I use more than anything else.

Performing actions in a loop over the output of another command.

There are different ways to do this, the most compact being xargs, but I prefer to use while read.

The main reason I like while over xargs is that xargs will die if you’re working with too many items since the entire operation happens on a single command-line action whereas while executes many separate actions.

This will make a whole lot more sense with some examples.

Here’s one I use all the time:

Deleting all .svn directories from a path (handy for making a working copy not be a working copy any more).

  find ./ -type d -name ".svn" | while read d; do rm -rf "$d"; done

Let’s break this down a little.

The first thing we’re doing is a simple find command that finds all directories (-type d) named .svn (-name “.svn”) in the current directory.

The next thing, and this is where all of the power of this construct comes from, is that we pipe the output of the find command to a Bash while loop. The while loop iterates over the find output and puts each value into the Bash variable $d (the name of the variable can be anything. I usually stick with something like ‘d’ for directories or ‘f’ for files). Then, for each time through the loop, we delete the directory in the $d variable.

The basic form of the while loop looks like this:

  while read d; do <stuff happens here> done

What you do within the while loop is entirely up to you. All of the usual Bash concepts for stringing commands together (using ;, &, or |) still work.

The real power of this idea is that you can do use it with anything that has newline-separated output.

Let’s say you have a file that contains a list of files like this:

foo_file1.txt
foo_file2.txt
bar_file1.txt
foo_file3.txt
foo_file4.txt
bar_file2.txt

Maybe we want to make all files that start with “bar_” world-writable.

  egrep "^bar_" filelist.txt | while read f; do chmod a+w "$f"; done

The egrep command gives us all lines from the file that start with “bar_”. That output gets piped to the while loop which does the chmod command.

Now a real world example.

My previous job involved dealing with massive amounts of data from an online survey. Of the many fields entered as part of this survey, one field was the price people paid for something. We had some validation on the price field but, one quarter, a bogus (and very large) price snuck through into the final data. This was a large problem because we generated a lots reports that used the Average Price Paid and this number was throwing them all off. We need to (quickly) go in and update all records.

Unfortunately, this involved checking and updating more tables than a sane person would ever want to deal with by hand.

So it was while loop to the rescue!

Step one is construct a command-line that will loop over all of the tables in a database. We have the egrep call to strip out the first line of the output which isn’t actually a table name. This example just prints out the name of each table.

  mysql -B -e "show tables" thedb | egrep -v "^Tables_in_" | while read tbl; do echo "$tbl"; done

Step two does a SELECT on each table and prints all of the ids with an invalid price. Once again, we do a grep to strip out some extra info that mysql is printing out.

  mysql -B -e "show tables" thedb | egrep -v "^Tables_in_" | while read tbl; do mysql -B -e "SELECT id FROM $tbl WHERE price >= 10000" | egrep "^[0-9]"; done

That’s the Bash trick I use the most. Remember, it works for anything that has newline-separated output.

I’m not to going to cover all of the PHP array functions but I thought I’d go over a few. I see a lot of code examples online that could be easily be shortened by taking advantage of these functions.

So here are a few interesting ones that might save you some work down the road.

array_map

This applies a callback function to every element of an array and returns the resulting array.

array array_map ( callback $callback , array $arr1 [, array $... ] )

For example: Applying the trim function to every element of an array.

$a = array(' a ', '   b', '   c   ');
$a = array_map('trim', $a);
print_r($a);

array_flip

This exchanges all keys with their associated values.

array array_flip ( array $trans )

$a = array( 'foo' => 1, 'bar' => 2, 'baz' => 3);
$b = array_flip($a);
print_r($b);

Array
(
[1] => foo
[2] => bar
[3] => baz
)

usort

This sorts an array by value but uses a custom function for the comparison which is handy if you have an array of objects and you want to sort by some property within those objects. According to the PHP docs, “The comparison function must return an integer less than, equal to, or greater than zero if the first argument is considered to be respectively less than, equal to, or greater than the second.”

bool usort ( array &$array , callback $cmp_function )

(To do the same thing by keys instead of values, use uksort)

function cmp_func($a, $b) {
    if( $a->val < $b->val ) {
        return -1;
    }elseif( $a->val == $b->val ) {
        return 0;
    }elseif( $a->val > $b->val ) {
        return 1;
    }
}

class Foo {
    public $val;

    public function __construct($val) {
        $this->val = $val;
    }
}

$a = array();

$a[] = new Foo(3);
$a[] = new Foo(2);
$a[] = new Foo(1);

echo '
' . print_r($a, true) . '
';
usort($a, 'cmp_func');
echo '
' . print_r($a, true) . '
';

array_slice

Extracts a section of an array.

array array_slice ( array $array , int $offset [, int $length [, bool $preserve_keys= false ]] )

$a = array( 'a', 'b', 'c', 'd');

$b = array_slice($a, 2);

echo '
' . print_r($b,true) . '
';

Array
(
[1] => c
[2] => d
)

array_sum

This returns the sum of the values in an array.

number array_sum ( array $array )

$a = array(1,2,3,4);

echo array_sum($a);

list

Assign values from an array to individual variables.

void list ( mixed $varname [, mixed $... ] )

$a = array( 'a', 'b', 'c');

list($val1, $val2, $val3) = $a;

echo "$val1, $val2, $val3";

a, b, c