Linux bash for find

How to loop over directories in Linux?

I am writing a script in bash on Linux and need to go through all subdirectory names in a given directory. How can I loop through these directories (and skip regular files)? For example:
the given directory is /tmp/
it has the following subdirectories: /tmp/A, /tmp/B, /tmp/C I want to retrieve A, B, C.

Solid array under bash, (accepting spaced dirnames): dirs=(/tmp/*/);dirs=(«$»);dirs=(«$») , then printf «%s\n» «$» or for dir in «$» ;do .

11 Answers 11

All answers so far use find , so here’s one with just the shell. No need for external tools in your case:

for dir in /tmp/*/ # list directories in the form "/tmp/dirname/" do dir=$ # remove the trailing "/" echo "$" # print everything after the final "/" done 

Well, yes, ‘find’ is kind of the Swiss Army knife on *NIX machines to find something file related. But a pure version with bash builtins only is good to know, too. +1

It is because the variable dir will include the backslash at the end. He is just removing it to have a clean name. % will remove the / at the end. ## will remove the / at the beginning. These are operators to handle substrings.

If there’s no match, the loop will be triggered with /tmp/*/ , it would be wise to include a check to see if the directory actually exists.

cd /tmp find . -maxdepth 1 -mindepth 1 -type d -printf '%f\n' 

• find finds files (quite obviously)
• . is the current directory, which after the cd is /tmp (IMHO this is more flexible than having /tmp directly in the find command. You have only one place, the cd , to change, if you want more actions to take place in this folder)
• -maxdepth 1 and -mindepth 1 make sure that find only looks in the current directory and doesn’t include . itself in the result
• -type d looks only for directories
• -printf ‘%f\n prints only the found folder’s name (plus a newline) for each hit.

By the way: Note, that all answers will find hidden folders, too (that is, folders starting with a dot)! If you don’t want this, add ` -regex ‘\./[^\.].*’` before the printf or exec options.

Helpful — if you need to execute only one command for each hit 😉 — Only I have several commands to execute 🙁 .

No problem: Adapt this solution: transnum.blogspot.de/2008/11/… Inside the while..done loop you can go crazy.

This is a very nice method for some use cases; find ‘s -exec option lets you run any command for each file/directory.

You can loop through all directories including hidden directrories (beginning with a dot) with:

for file in */ .*/ ; do echo "$file is a directory"; done 

note: using the list */ .*/ works in zsh only if there exist at least one hidden directory in the folder. In bash it will show also . and ..

Another possibility for bash to include hidden directories would be to use:

shopt -s dotglob; for file in */ ; do echo "$file is a directory"; done 

If you want to exclude symlinks:

for file in */ ; do if [[ -d "$file" && ! -L "$file" ]]; then echo "$file is a directory"; fi; done 

To output only the trailing directory name (A,B,C as questioned) in each solution use this within the loops:

file="$" # strip trailing slash file="$" # strip path and leading slash echo "$file is the directoryname without slashes" 

Example (this also works with directories which contains spaces):

mkdir /tmp/A /tmp/B /tmp/C "/tmp/ dir with spaces" for file in /tmp/*/ ; do file="$"; echo "$"; done 

Works with directories which contains spaces

while IFS= read -d $'\0' -r file ; do echo $file; ls $file ; done < <(find /path/to/dir/ -mindepth 1 -maxdepth 1 -type d -print0) 

Original post (Does not work with spaces)

Inspired by Boldewyn: Example of loop with find command.

for D in $(find /path/to/dir/ -mindepth 1 -maxdepth 1 -type d) ; do echo $D ; done 

find . -mindepth 1 -maxdepth 1 -type d -printf "%P\n" 

The technique I use most often is find | xargs . For example, if you want to make every file in this directory and all of its subdirectories world-readable, you can do:

find . -type f -print0 | xargs -0 chmod go+r find . -type d -print0 | xargs -0 chmod go+rx 

The -print0 option terminates with a NULL character instead of a space. The -0 option splits its input the same way. So this is the combination to use on files with spaces.

You can picture this chain of commands as taking every line output by find and sticking it on the end of a chmod command.

If the command you want to run as its argument in the middle instead of on the end, you have to be a bit creative. For instance, I needed to change into every subdirectory and run the command latemk -c . So I used (from Wikipedia):

find . -type d -depth 1 -print0 | \ xargs -0 sh -c 'for dir; do pushd "$dir" && latexmk -c && popd; done' fnord 

This has the effect of for dir $(subdirs); do stuff; done , but is safe for directories with spaces in their names. Also, the separate calls to stuff are made in the same shell, which is why in my command we have to return back to the current directory with popd .

Источник

Bash script for loop with find and many directories

I'm writing a bash script to do some work on a directory that contains many (100,000+) sub-directories. Is there a predefined limit to how many arguments you can pass to a for loop such as the following?

for dir in $(find . -type d) do # My code done 

Out of interest, what filesystem are you using? I was under the impression that ext3, for example, was limited to 32,000 directories.

This is an ext4 filesystem. The directories are not all in one dir, rather they are organized by md5 in subdirs, e.g. 2/f/f80d0642f8ebfb07d23547adc107cb40

4 Answers 4

Don't parse find output, and just use shell globbing - it's safer and built into the shell. Shell built-ins like for are not subject to the same argument list length limit as external processes since no calls do exec* are made.

I actually omitted this because I didn't think it mattered, but I need to find only directories modified within a certain time frame, e.g. find . -type d -mtime 1. Is this possible with your syntax?

@Magnus Either use find -exec or check the time in the for loop. Just don't try to parse the output of find or anything that simply prints file names.

Is there a source you can point to regarding why this would be a bad idea? I just tried a for..in $(find) with 40,000 directories and it seems to work fine.

Unix file names are allowed to contain any character except \0 and / . Things like that are basically binary blobs and can't be reliably parsed using things like command substitution - too many possible funny characters to mess up the parsing. Try mkdir test && cd test && mkdir 'one dir' && for dir in $(find . -type d); do printf '%s\n' "$dir"; done . You should see two directories, one and dir .

Ah, that's the point you were making. However, I control exactly the files in this directory, so I know they will not contain any "funny" characters. I guess my question was more "Will a bash for loop blow up if the find list becomes too large".

If you have bash 4 , you can do something recursive like :

shopt -s globstar for dir in **/; do echo "$dir"; done 

I don't agree with the answer you accepted -- you will run into memory issues if you use $(find) , even without an exec present.

find . -type d | while IFS= read -r dir do # My code done 

(Note: this assumes that there is no directory name containing a newline character.)

Then you won't need the temporary memory to store the find output as you would with the command substitution. This would also work if the find or other command never terminated, e.g.:

# will not work! for line in $(yes) ; do echo "$line" ; done # works yes | while IFS= read -r line ; do echo "$line" ; done 

So your solution runs the while loop for each line of output as soon as each output line from find appears, while mine pre-computes the entire thing and then runs the for loop. If I'm understanding things correctly. Surely the find will spit out dirs faster than the while loop can process them, so it would still need to buffer the output somewhere.

Yes, your first sentence is correct. For the second, no, the pipe will block and so find will also block while writing its output. In other words, find will simply wait if the loop can't keep up.

Wow, that is neat. I didn't even know pipes could block. You learn something new every day! I have just rewritten my script to do it your way and avoid any SNAFUs. I regret that I have but one accepted answer to give.

Don't generate file names with a command substitution: this will break if the file names contain whitespace or \[?* .

With bash ≥4, ksh93 or zsh, you can avoid most uses of find that only use -type d or -name … predicates by using the ** glob to match subdirectories at any depth. In bash, run shopt -s globstar first. In ksh, run set -o globstar first.

Portably, or in cases where you need more, use find … -exec … + or find … -print0 | xargs -0 … . This approach has the added benefit of running find somewhat in parallel with the action on the files, though this only becomes true for huge directory trees (at least thousands of matching files). You can make find or xargs run a shell if you need to do more than run one command.

find -type d -exec sh -c 'for x in "$@"; do echo "$x"; done' _ <> + 

(That _ is $0 in the shell snippet, which we don't use.)

Источник

Bash — for и find

2. Написать скрипт с использованием цикла for, выводящий на консоль размеры и права доступа для всех файлов в заданном каталоге и всех его подкаталогах (имя каталога задается пользователем в качестве первого аргумента командной строки).

Как именно связать цикл for с результатом из find ?

#!/usr/bin/env bash for file in `find $1 -type f`; do ls -ogh --time-style=+"" $file done

Уже лучше, но есть замечания:
- Как выводить в такой очерёдности: $file РАЗМЕР ПРАВА_ДОСТУПА

++ Когда find не имеет доступа (к примеру find /etc не из под root) - выводит: find: /etc/. Ошибка, нет доступа . а как в выводе заменить на: "текущий скрипт" Ошибка, нет доступа?

man find
man ls
man grep
man awk
google.com

А вы зачем вопрос задали? чтобы за вас задачу из универа решили? Гнать вас в шею оттуда.
Это ресурс не о том, чтобы решать чужие задачи. Это ресурс о вопросах и знаниях.

А кто сказал, что задача не решена? Не получантся единствннный момент (замена слова). А вы, если не знаете пл сути вопроса - воздержитесь от своих умных комментариев.

for file in `find . `; do echo $file; done

А теперь внимание: так задачи решать не надо! В реальной ситуации вам могут попасться файлы с пробелами или ещё хуже. Правильно задача решается вот так:

find ./dir -print0 | xargs -0 stat

find и xargs поставляются в одном пакете, а -print0 гарантирует, что имена файлов при передаче никак не пострадают. Плюс советую man find, опция -exec.

А если нужно использовать ИМЕННО for, + нужно вывести только размер, права доступа и имя файла. ++ Очень важно - в результате заменить слово find: ( в контексте оно будет find: . отказано в доступе) - на название исполняющегося скрипта ($0) . как тут быть, подскажите?

Войдите, чтобы написать ответ

Безопасен ли линукс для хранения файлов?

Источник