Recursively look for files with a specific extension
I’m trying to find all files with a specific extension in a directory and its subdirectories with my Bash (Ubuntu 10.04 LTS (Lucid Lynx) release). This is what’s written in a script file:
#!/bin/bash directory="/home/flip/Desktop" suffix="in" browsefolders () for i in "$1"/*; do echo "dir :$directory" echo "filename: $i" # echo $ extension=`echo "$i" | cut -d'.' -f2` echo "Erweiterung $extension" if [ -f "$i" ]; then if [ $extension == $suffix ]; then echo "$i ends with $in" else echo "$i does NOT end with $in" fi elif [ -d "$i" ]; then browsefolders "$i" fi done > browsefolders "$directory" [: 29: in: unexpected operator (with $extension instead of ‘in’ ) What’s going on here, and where’s the error? But this curly brace.
10 Answers 10
find "$directory" -type f -name "*.in" is a bit shorter than that whole thing (and safer. It deals with whitespace in filenames and directory names).
Your script is probably failing for entries that don’t have a . in their name, making $extension empty.
@flip: that’s a different question. Post a new question, detailing exactly what you’d like to do and what you’ve tried so far.
One little correction: use ‘*.in’ or \*.in instead of «*.in» because double quotes don’t prevent shell expansion. I.e. your script will not work properly if there’s a file with .in extension in the current directory.
find -type f -name '*.extension' Example: To find all csv files in the current directory and its sub-directories, use:
The syntax I use is a bit different than what Mat suggested:
find $directory -type f -name \*.in Matt’s script also won’t work if there’s a file with .in extension in the current directory, while yours would still work. See stackoverflow.com/questions/5927369/…
@gniourf_gniourf You should provide some reference for your statement, otherwise one could simply argue: «No, you are wrong». But in fact you’re right: gnu.org/software/bash/manual/html_node/Double-Quotes.html
@user1885518: I think that it should be the guy who claims that the script doesn’t work who should provide some examples where the script fails. That’s what I do when I leave comments where there are broken scripts: it’s usually about quotes and filenames containing spaces, newlines, globs, etc., and I specifically explain why it’s broken.
Providing reference is always a good way in a discussion, it does not depend on who was the first. He should, you should.
du -a $directory | awk '' | grep '\.in$' The grep isn’t really necessary here. awk has regular expressions and could limit its output to values matching a pattern.
This method is extremely useful if your going through 100s of terabyte. Find command takes too much time to process. This starts immediately.
Though using find command can be useful here, the shell itself provides options to achieve this requirement without any third party tools. The bash shell provides an extended glob support option using which you can get the file names under recursive paths that match with the extensions you want.
The extended option is extglob which needs to be set using the shopt option as below. The options are enabled with the -s support and disabled with he -u flag. Additionally you could use couple of options more i.e. nullglob in which an unmatched glob is swept away entirely, replaced with a set of zero words. And globstar that allows to recurse through all the directories
shopt -s extglob nullglob globstar Now all you need to do is form the glob expression to include the files of a certain extension which you can do as below. We use an array to populate the glob results because when quoted properly and expanded, the filenames with special characters would remain intact and not get broken due to word-splitting by the shell.
For example to list all the *.csv files in the recursive paths
The option ** is to recurse through the sub-folders and *.csv is glob expansion to include any file of the extensions mentioned. Now for printing the actual files, just do
Using an array and doing a proper quoted expansion is the right way when used in shell scripts, but for interactive use, you could simply use ls with the glob expression as
This could very well be expanded to match multiple files i.e. file ending with multiple extension (i.e. similar to adding multiple flags in find command). For example consider a case of needing to get all recursive image files i.e. of extensions *.gif , *.png and *.jpg , all you need to is
This could very well be expanded to have negate results also. With the same syntax, one could use the results of the glob to exclude files of certain type. Assume you want to exclude file names with the extensions above, you could do
excludeResults=() excludeResults=(**/!(*.jpg|*.gif|*.png)) printf '%s\n' "$" The construct !() is a negate operation to not include any of the file extensions listed inside and | is an alternation operator just as used in the Extended Regular Expressions library to do an OR match of the globs.
Note that these extended glob support is not available in the POSIX bourne shell and its purely specific to recent versions of bash . So if your are considering portability of the scripts running across POSIX and bash shells, this option wouldn’t be right.
How to Search and Find Files Recursively in Linux
This brief tutorial explains how to search and find the files recursively in the Linux operating systems.
After reading this article, you will be able to find any file recursively using the different techniques including a single file search, multiple files search, find files by permissions, and more. This document is optimized for both new and experienced Linux users. All methods are valid for every Linux distribution.
All examples in this tutorial contain screenshots to make it easy for any Linux user to understand and reproduce them.
Finding Files Recursively in Linux
The find command does not need flags to search the files recursively in the current directory. You only need to define the main directory and the file name using the –name option. This command will search the file within the main directory and all subdirectories.
The syntax is simple, as shown in the following:
If you want to find the 27.jpg file within your home directory and subdirectories, run the following command as shown in the following screenshot:
As you can see, the file was found in the /home/linuxhint/Downloads/recur subdirectory.
An alternative to the previous command is the tree command shown in the following example where you search the same file named 27.jpg within the current directory:
As you can see in the previous figure, the format is pretty different. It seems to be more user friendly or nicer as long as you don’t need to copy the full path to paste it.
The following screenshot shows how to use the find command to recursively search more than a file.
The syntax to search multiple files recursively is the following:
Note that there is a –o flag before the second file name. You can add more than one file by adding more –o –name flags. For example, to find 4 files with the same command, use the following syntax:
In the practical example described in the following image, use this command to find a file named 27.jpg and a file whose name begins with “DIAGRAM” but without specifying its extension. Instead, use a wildcard (*) to find any file named DIAGRAM independently of its type.
As you can see in the previous image, both files were found recursively.
The next example describes how to find the files by extension using the find command. In the following figure, you can see how to recursively find all the .jpg files using the wildcard again. The syntax is pretty simple:
Thus, to find all the .jpg files recursively, run the following command:
As shown in the previous image, all the jpg files including their path are listed successfully. You can replace the .jpg extension for any extension that you want to search like .png, .txt, .c and more.
Now, let’s assume that you don’t want to find a file but a directory recursively. All you need to do is to use the same command that was shown in the first example of this tutorial then add the -type d option. The syntax as follows:
In the following practical example, use the previous syntax to find the recur directory.
As you see in the previous figure, the directory named “recur” was found successfully.
You also can find the files by size using the following syntax where is the main directory containing the subdirectories and the is the size of the files that you can list with their full path.
The following example describes how to find the 10 MB size files. You can replace the M defining units in MB with c for bytes, w for two two byte words, k for kibytes and G for gibibytes (note units are case sensitive).
To find the 10 mebibytes files, execute the following command:
All 10M files were properly listed with their paths.
The syntax to find the files based on their permissions is shown in the following:
Let’s assume that you want to identify and list the files with read, write, and executing permissions (777). The command to run is the following:
The last example of this tutorial shows how to find and list the files and directories by size.
As shown, the files are listed by size with proper units. The 0 size directories and files are empty.
Conclusion
Linux versatility and flexibility allows to find the files (and other functions) recursively in many ways. They can easily be executed by all the Linux users independently of his knowledge level, from the new users to the system administrators. All techniques previously described are valid for all the Linux distributions and even to some Unix systems. According to their man pages, some flags may vary in some distributions, but most of them are universal. In case your Linux distribution does not match any of the previously explained commands, you can read the man page. It is highly recommended to the readers to practice the examples to incorporate this knowledge.
Thank you very much for reading this Linux tutorial. Keep following us for more Linux professional tips.
About the author
David Adams
David Adams is a System Admin and writer that is focused on open source technologies, security software, and computer systems.
Linux Find Recursive
Recursive directory listing allows you to view and find files that are nested inside other subdirectories.
One of the commands that are built for that purpose is the tree command. A tree is a simple and powerful directory listing utility. You can specify the level of depth that you wish to show in a single command.
Tree, as the name suggests, allows you to show files in nested directories in a tree-like format. It also gives you details about the specified path, including the total number of files and directories, etc.
To install it, use your system package manager:
You can then use the tree command followed by the path to the target directory.
For example, the command below shows all the files and directories inside the /var/logs directory:
To find all the files and directories, including hidden files, you can use the -a flag with the tree command:
Using Find
You can also use the find command followed by the target directory and the file you wish to locate.
For example, to locate the file access.logs in the root directory, use the command:
The command will start in the root directory and recursively search all the subdirectories and locate any file with the specified name.
To find any files ending with a specific extension, run the command:
The command will start in the current working directory as specified by the period and recursively search for all files ending with the .txt extension.
Using fd Command
The fd command is a free, open-source utility that is developed as an alternative to the find command.
It is very fast and user-friendly and has incredible functionality. It’s a great tool when you need to locate a file that is nested inside a series of subdirectories.
To install it, run the commands: