rm: Argument list too long
Posted by davidnewcomb on 09 Aug 2008 in Techie
Occasionally, (under Unix) I want to delete all the files in a directory, but when I issue the rm -rf * command it gives me the error:
Most of the time you just want to delete all the files in the directory without any trouble. The quickest and easiest is to write a simple shell script to do the job.
-bash: /bin/rm: Argument list too longThis happens because the star is interpreted by the shell as:
rm -rf file1.txt file2.txt …etc… file10000.txtIf the character length of the files in the directory (plus the length of “rm -rf “, plus a space between each file name) is greater then the system’s maximum length of a shell command line, then you will get the error message: Argument list too long. There are several ways you can achieve your objective. The first is the simplest, you can decrease the amount of files in your command line. To do this narrow the wild-card expression to something tighter. So in our example try:
rm -rf file[1-5]*.txt rm -rf file[6-9]*.txtThis has split the command line in half. If that is not enough try:
rm -rf file1*.txt : rm -rf file9*.txtto split them into tenths. Sometimes this can take quite a while to find the right combination because you always want to do it in the minimum number of commands so you tend to try the larger subsets first - which usually fail for the same reason. You will have to know the nature of the file names in the directory, so in some cases like log files this is easy, but for others such as temporary folders it can be trickier. On some occasions you know that the files in the directory are chronologically ordered, like a folder of web logs. In this instance you may want to selectively delete the files according to their date. There is a very handy Unix command called, funnily enough, find which can help you find files based on all sorts of characteristics. In this case date.
find . -mtime 30 -print -exec rm -rf {} \;This will delete the files older than 30 days and print them on the screen as it finds them.
. | from the current directory |
-mtime 30 | modification time in days |
prints the file found on the console (optional) | |
-exec rm -rf {} \; | each file found is substituted into the {} and the ‘;’ has been delimited so that the shell doesn’t take it as a end-of-statement marker |
ls | while read line; do rm -f “$line"; doneThis will read the result of the list files command (ls) and send it one line at a time into a loop. The loop runs the delete file (rm) command specifying the file from the directory. Job done! If you have to do this a lot it would be really handy to have a permanent copy of this instruction, as a command in your system’s administration arsenal. So here is what you might have.
#!/bin/sh
#
# (c) BigSoft Limited 2008
# Please display this copyright message
# Feel free to use.
# Program comes without any warranty
usage ()
{
echo "purpose: delete all files in directory"
echo "usage: bigdel [-h] [-l] [-d] [-f all|n] [directory]"
if [ "$1" != "" ]
then
echo "reason: $1"
fi
echo ""
echo "where:"
echo " -h help"
echo " -l list files to be deleted"
echo " -d do the delete"
echo " -f all filter: delete all files"
echo " -f n filter: delete files older than 'n' days"
echo "directory directory you want to delete the files in"
echo " or current directory is not specified"
echo ""
echo "The default behaviour is to count the number of files to be deleted"
if [ "$1" = "" ]
then
exit 0
fi
exit 1
}
# Init
DIR_NAME=
IS_NUMBER=
DELETE_DAYS=all
LIST_ONLY="no"
DO_DELETE="no"
FIND_OPT=
# Parse command line args
# Colon at the front of getopts prevents warning messages
while getopts ":hld:f:" SWITCH $*
do
# Help
if [ "$SWITCH" = "h" ]
then
usage
exit 0
fi
# Help or bad parameter
if [ "$SWITCH" = "?" ]
then
usage "Unknown switch"
exit 0
fi
if [ "$SWITCH" = "l" ]
then
LIST_ONLY="yes"
fi
if [ "$SWITCH" = "d" ]
then
DO_DELETE="yes"
fi
# Style of Directory delete
if [ "$SWITCH" = "f" ]
then
echo "$OPTARG" | egrep "^[0-9]+$" >/dev/null 2>&1
IS_NUMBERS=$?
if [ "$OPTARG" != "all" -a "$IS_NUMBERS" != "0" ]
then
usage "-f can only be 'all' or a number of days"
exit 0
fi
DELETE_DAYS=$OPTARG
fi
done
# Move argument pointer to next non switched argument
OPTIND=`expr $OPTIND - 1`
shift $OPTIND
if [ "$2" != "" ]
then
usage "Unknown extra argument"
exit 0
fi
if [ "$1" = "" ]
then
DIR_NAME="."
else
DIR_NAME=$1
if [ ! -d "$DIR_NAME" ]
then
usage "$DIR_NAME is not a directory"
exit 0
fi
fi
if [ "$DELETE_DAYS" != "all" ]
then
FIND_OPT="-mtime +$DELETE_DAYS"
fi
if [ "$LIST_ONLY" = "yes" ]
then
find $DIR_NAME $FIND_OPT -print
fi
if [ "$DO_DELETE" = "yes" ]
then
find $DIR_NAME $FIND_OPT -exec rm {}\;
fi
# Print report
TOTAL=`find $DIR_NAME $FIND_OPT -print | wc -l`
echo "There are $TOTAL file(s) to delete"
exit 0
2 comments
Comment from: shiben [Visitor]

Thanks for the bigdel script! I found several ways to do this same thing, some of which you list, but was very happy to find someone who created a permanent script to solve this problem. Cheers!
Comment from: davidnewcomb [Member]

I’ll be interested in what circumstances anyone is using it in.
Form is loading...