The Wonders of xargs

Exploring the goodness of xargs, one of the most versatile unix utilities.

Posted on July 31, 2011 unix, bash .

xargs is a nice little utility for the command line that I use quite a lot. The basic usecases are pretty simple and it might even be so simple that the wonders of xargs go unnoticed for quite some time.
xargs is useful whenever you want to apply a command to a list of items in the shell. One really simple example:

$ find . -type f | xargs md5

will calculate the md5 checksum of each file in the current directory:

MD5 (1.txt) = dfb5cfe29927bb659c380f553eb43131
MD5 (2.txt) = fd97581244bf2ff3241f977c432f2c0b
MD5 (3.txt) = 63a79bfa8ff94b772f16a6fea311b9a7
MD5 (4.txt) = 2e848e59c3a2abe9dd0fe250af7ac2dc
MD5 (5.txt) = 2098d6fce034058709de9120bd366bee

Here xargs will simply take the output of the find command and feed it to the md5 command.
The output will be fed as one chunk so the md5 command will be invoked like this:

$ md5 1.txt 2.txt 3.txt 4.txt 5.txt

If you want xargs to split up the string and feed it one by one to the md5 command, you would need to use the -n option:

$ find . -type f -print0 | xargs -0 -n1 md5

Note that xargs will take any string and by default interpret spaces, tabs and newlines as delimiters.

A little more complicated form shows how xargs can take a command where the first parameter is already bound.

$ find . -name "*.h" | xargs grep assert

Up to here it’s pretty straight forward since we only had to supply the the last parameter to the command after xargs. But it’s a little more involved when the parameter is not a the end, e.g. in a copy command:

$ mkdir tmp
$ find . -type f -name "*.txt" | xargs -I xxx cp xxx tmp

This will find all files ending in “.txt” and copy them into a temporary directory.
Here the -I option allows to define a replacement string for the following command. So each input is bound to the variable xxx and can then be used in the copy command.
Note that this will not work for files that contain spaces! To make it work with such nasty filenames we have to use the -0 option:

$ find . -type f -name "*.txt" -print0 | xargs -0 -I xxx cp xxx tmp

One very useful option is -p. This will prompt the user before executing each command invocation which sometimes is what you want.

$ find old -type f -name "*.txt" -print0 | xargs -0 -n1 -p rm

will find all textfiles in the old directory, feed each of these to xargs (using the -n1 option) and remove them after prompting the user.

It’s good to know about xargs and how to use it… It definitely WILL come in handy in lot’s of situations.
One final example (‘cause it’s kind of a favorite of mine):

$ find . -type d -name '.svn' -print0 | xargs -0 rm -rdf

Will delete all the @*#!%-.svn folders recursivly from the current directory! Gone and forever!
Long live the git!