I needed to copy files generated by doxygen from one directory into another for a large opensource C++ project. Sadly there were too many files in the directory, so bash started complaining 🙁 cp and rsync died out with the error of argument list too long. initially I figured I could generate it all from scratch in new location but it was quicker and easier to use a for loop to rsync the files over 🙂

some info:

• all files start with alphabetic characters.
• there are no spaces in the names
• all files are in single directory

I realized bash expansion would work here.

Using for loop

for x in {a..b}
do
    echo $x*
done

Notice I only stepped between A and B because I didn’t want to sit there for an hour while it listed all the files. this worked well, it listed all files and I was sure it would suite my purposes. now the real deal!

for x in {a..z}
do
echo $x
rsync -az /backups/doxygen/$x* /home/user/current/directory/
done

sometimes you might still get the error even for each letter, for example I still had too many files starting with D and Q. so I just changed where I globbed :

for x in {a..z}
do
echo $x
rsync -az /backups/doxygen/d$x* /home/user/current/directory/
done

this allows me to further iterate a thru z but after starting the files with the letter d. Now what happens if you happen to have files starting with numbers? simply switch the letters for numbers.

for x in {0..9}
do
echo $x
rsync -az /backups/doxygen/$x* /home/user/current/directory/
done

You can use any other command you need in place of rsync. like mv cp mkdir or any custom commands.

for x in {a..z}
do
echo $x
mv /backups/doxygen/$x* /home/user/current/directory/
done

Globbing

Now if you don’t want to use for loops you can glob them in a one liner like so :

ls /backups/doxygen/[x-z]*

and the actual command using cp and globbing

cp -r /backups/doxygen/[a-z]* /home/user/current/directory/

and again going a level deeper

cp -r /backups/doxygen/d[a-z]* /home/user/current/directory/

Voila! argument list too long is now vanquished! do any of you have a better way of dealing with this? let me know!

More info about globbing.