Recursively look for files with a specific extension
I’m trying to find all files with a specific extension in a directory and its subdirectories with my Bash (Ubuntu 10.04 LTS (Lucid Lynx) release). This is what’s written in a script file:
#!/bin/bash directory="/home/flip/Desktop" suffix="in" browsefolders () for i in "$1"/*; do echo "dir :$directory" echo "filename: $i" # echo $ extension=`echo "$i" | cut -d'.' -f2` echo "Erweiterung $extension" if [ -f "$i" ]; then if [ $extension == $suffix ]; then echo "$i ends with $in" else echo "$i does NOT end with $in" fi elif [ -d "$i" ]; then browsefolders "$i" fi done > browsefolders "$directory" [: 29: in: unexpected operator (with $extension instead of ‘in’ ) What’s going on here, and where’s the error? But this curly brace.
10 Answers 10
find "$directory" -type f -name "*.in" is a bit shorter than that whole thing (and safer. It deals with whitespace in filenames and directory names).
Your script is probably failing for entries that don’t have a . in their name, making $extension empty.
@flip: that’s a different question. Post a new question, detailing exactly what you’d like to do and what you’ve tried so far.
One little correction: use ‘*.in’ or \*.in instead of «*.in» because double quotes don’t prevent shell expansion. I.e. your script will not work properly if there’s a file with .in extension in the current directory.
find -type f -name '*.extension' Example: To find all csv files in the current directory and its sub-directories, use:
The syntax I use is a bit different than what Mat suggested:
find $directory -type f -name \*.in Matt’s script also won’t work if there’s a file with .in extension in the current directory, while yours would still work. See stackoverflow.com/questions/5927369/…
@gniourf_gniourf You should provide some reference for your statement, otherwise one could simply argue: «No, you are wrong». But in fact you’re right: gnu.org/software/bash/manual/html_node/Double-Quotes.html
@user1885518: I think that it should be the guy who claims that the script doesn’t work who should provide some examples where the script fails. That’s what I do when I leave comments where there are broken scripts: it’s usually about quotes and filenames containing spaces, newlines, globs, etc., and I specifically explain why it’s broken.
Providing reference is always a good way in a discussion, it does not depend on who was the first. He should, you should.
du -a $directory | awk '' | grep '\.in$' The grep isn’t really necessary here. awk has regular expressions and could limit its output to values matching a pattern.
This method is extremely useful if your going through 100s of terabyte. Find command takes too much time to process. This starts immediately.
Though using find command can be useful here, the shell itself provides options to achieve this requirement without any third party tools. The bash shell provides an extended glob support option using which you can get the file names under recursive paths that match with the extensions you want.
The extended option is extglob which needs to be set using the shopt option as below. The options are enabled with the -s support and disabled with he -u flag. Additionally you could use couple of options more i.e. nullglob in which an unmatched glob is swept away entirely, replaced with a set of zero words. And globstar that allows to recurse through all the directories
shopt -s extglob nullglob globstar Now all you need to do is form the glob expression to include the files of a certain extension which you can do as below. We use an array to populate the glob results because when quoted properly and expanded, the filenames with special characters would remain intact and not get broken due to word-splitting by the shell.
For example to list all the *.csv files in the recursive paths
The option ** is to recurse through the sub-folders and *.csv is glob expansion to include any file of the extensions mentioned. Now for printing the actual files, just do
Using an array and doing a proper quoted expansion is the right way when used in shell scripts, but for interactive use, you could simply use ls with the glob expression as
This could very well be expanded to match multiple files i.e. file ending with multiple extension (i.e. similar to adding multiple flags in find command). For example consider a case of needing to get all recursive image files i.e. of extensions *.gif , *.png and *.jpg , all you need to is
This could very well be expanded to have negate results also. With the same syntax, one could use the results of the glob to exclude files of certain type. Assume you want to exclude file names with the extensions above, you could do
excludeResults=() excludeResults=(**/!(*.jpg|*.gif|*.png)) printf '%s\n' "$" The construct !() is a negate operation to not include any of the file extensions listed inside and | is an alternation operator just as used in the Extended Regular Expressions library to do an OR match of the globs.
Note that these extended glob support is not available in the POSIX bourne shell and its purely specific to recent versions of bash . So if your are considering portability of the scripts running across POSIX and bash shells, this option wouldn’t be right.
Find All Files with Extension in Linux
Often, we find ourselves stuck when we have to find all files with the same or different extensions. This has most likely happened to various Linux users while using the terminal. It is one thing to search for a single file type or file, but what will you do when you want to find out all files simultaneously? This article comes to the rescue for our readers who have such a dilemma.
We can use various Linux utilities for finding or locating files on a file system, but searching all files or filenames with the same or different extensions can be difficult and require specific patterns or expressions. In the upcoming section of the article, we will understand the working, syntax, and execution of these utilities.
Find command
One of the most powerful file searching tools in the Linux system is the “find command.” It searches the entire directory for files and folders to get matched with the user’s expression and performs actions on these files. File permission, file size, type are some other factors based on finding files on Linux. Find command also be combined with other utilities such as sed or grep. Now, lets’ head towards the practical implication of find command.
Find command syntax:
Finding all files with a single extension:
To find all files with a file extension, write out its path to find a command with the options and expression specifying the extension. In the below-given example, we will find all files with the “.txt” extension.
“.” in this command denotes that this tool will find all the “.txt” files in the current directory.
Find “.exe” files in the same find command by adding the extension as “*exe.”
Configuration files are also an essential part of any file system that can be used for multiple purposes. Write out this command for searching configuration files in the current directory.
Finding files with multiple extension:
You can also add more than extension in your find command so that you can find several extension files easily and quickly.
The execution of below given command will retrieve files with extension “.sh” and “.txt”
Locate command
The locate command is a faster and better tool as compared with “find.” When a file is initiated, instead of searching it in the file system, locate utilize the database for the searching requirement. This database stores parts and bits of the information related to files and their addresses on your system.
locate command syntax:
Finding a file with a specific extension, such as “.conf,” which is considered in our case, adds the directory path where the process of searching files will occur.
Find configuration files in the present working directory by utilizing the below-given command.
Similarly, you can follow the syntax of locate command for finding all files with any specific extension such as “.txt.”
Conclusion:
This post covers two powerful yet simple utilities for you to find all files with the same or different extensions. We have provided you the fundamental concepts regarding the “find” and “locate” command and shown you how to utilize these two Linux command-line tools to find all files with several extensions.
About the author
Talha Saif Malik
Talha is a contributor at Linux Hint with a vision to bring value and do useful things for the world. He loves to read, write and speak about Linux, Data, Computers and Technology.
How to use find command to find all files with extensions from list?
@Gerald: It’s likely you need to group your OR expression in escaped parentheses: find /path/to/ \( -iname ‘*.gif’ -o -iname ‘*.jpg’ \) -exec ls -l <> \; otherwise the exec only applies to the last part ( -iname ‘*.jpg’ in this case).
That’s an important remark. find /path/to/ -iname ‘*.gif’ -o -iname ‘*.jpg’ -print0 will only print the jpg files! You need brackets here: find /path/to/ \( -iname ‘*.gif’ -o -iname ‘*.jpg’ \) -print0
find -E /path/to -regex «.*\.(jpg|gif|png|jpeg)» > log
The -E saves you from having to escape the parens and pipes in your regex.
Hmmm the -E option tells find to use «extended regular expressions». Several other tools have a similar option, but I’m not sure this option is available on all UNIX distributions.
@tboyce12 Working on ubuntu, I can specify the -regextype to simplify the regex expression: find . -regextype posix-extended -regex «.*\.(jpg|gif|png|jpeg)» .
@cjm Maybe find -E /path/to -iregex «.*\.(jpg|gif|png|jpeg)» > log . Using the -iregex flag tells find to match case insensitively.
find /path/to/ -type f -print0 | xargs -0 file | grep -i image This uses the file command to try to recognize the type of file, regardless of filename (or extension).
If /path/to or a filename contains the string image , then the above may return bogus hits. In that case, I’d suggest
cd /path/to find . -type f -print0 | xargs -0 file --mime-type | grep -i image/