Use find command but exclude files in two directories
I want to find files that end with _peaks.bed , but exclude files in the tmp and scripts folders. My command is like this:
find . -type f \( -name "*_peaks.bed" ! -name "*tmp*" ! -name "*scripts*" \) But it didn’t work. The files in tmp and script folder will still be displayed. Does anyone have ideas about this?
7 Answers 7
Here’s how you can specify that with find :
find . -type f -name "*_peaks.bed" ! -path "./tmp/*" ! -path "./scripts/*" Explanation:
- find . — Start find from current working directory (recursively by default)
- -type f — Specify to find that you only want files in the results
- -name «*_peaks.bed» — Look for files with the name ending in _peaks.bed
- ! -path «./tmp/*» — Exclude all results whose path starts with ./tmp/
- ! -path «./scripts/*» — Also exclude all results whose path starts with ./scripts/
Testing the Solution:
$ mkdir a b c d e $ touch a/1 b/2 c/3 d/4 e/5 e/a e/b $ find . -type f ! -path "./a/*" ! -path "./b/*" ./d/4 ./c/3 ./e/a ./e/b ./e/5 You were pretty close, the -name option only considers the basename, where as -path considers the entire path =)
Nice work. However, you forgot one of the things the OP wanted, to find files ending with _peaks.bed .
This uses a number of extension in GNU find , but since the question is tagged Linux, that is not a problem. Good answer.
A short note: if you use . at your initial find prompt, you have to use it in each path you exclude. The path matching is pretty strict, it doesn’t do fuzzy searching. So if you use find / -type f -name *.bed» ! -path «./tmp/» its not going to work. you need to have ! -path «/tmp» to make it happy.
According to the man pages: «To ignore a whole directory tree, use -prune rather than checking every file in the tree.» If your excluded directories run very deep or has tons of files and you care about performance, then use the -prune option instead.
find \( -path "./tmp" -o -path "./scripts" \) -prune -o -name "*_peaks.bed" -print find \( -path "./tmp" -o -path "./scripts" \) -prune -false -o -name "*_peaks.bed" find \( -path "./tmp" -path "./scripts" \) ! -prune -o -name "*_peaks.bed" The order is important. It evaluates from left to right. Always begin with the path exclusion.
Explanation
Do not use -not (or ! ) to exclude whole directory. Use -prune . As explained in the manual:
−prune The primary shall always evaluate as true; it shall cause find not to descend the current pathname if it is a directory. If the −depth primary is specified, the −prune primary shall have no effect. and in the GNU find manual:
-path pattern [. ] To ignore a whole directory tree, use -prune rather than checking every file in the tree. Indeed, if you use -not -path «./pathname» , find will evaluate the expression for each node under «./pathname» .
find expressions are just condition evaluation.
- \( \) — groups operation (you can use -path «./tmp» -prune -o -path «./scripts» -prune -o , but it is more verbose).
- -path «./script» -prune — if -path returns true and is a directory, return true for that directory and do not descend into it.
- -path «./script» ! -prune — it evaluates as (-path «./script») AND (! -prune) . It revert the «always true» of prune to always false. It avoids printing «./script» as a match.
- -path «./script» -prune -false — since -prune always returns true, you can follow it with -false to do the same than ! .
- -o — OR operator. If no operator is specified between two expressions, it defaults to AND operator.
Hence, \( -path «./tmp» -o -path «./scripts» \) -prune -o -name «*_peaks.bed» -print is expanded to:
[ (-path "./tmp" OR -path "./script") AND -prune ] OR ( -name "*_peaks.bed" AND print ) The print is important here because without it is expanded to:
-print is added by find — that is why most of the time, you do not need to add it in you expression. And since -prune returns true, it will print «./script» and «./tmp».
It is not necessary in the others because we switched -prune to always return false.
Hint: You can use find -D opt expr 2>&1 1>/dev/null to see how it is optimized and expanded,
find -D search expr 2>&1 1>/dev/null to see which path is checked.
This doesn’t seem to work without a -name predicate — i.e. when finding files by type with -type f . I get the error find: paths must precede expression.
@HashimAziz I cannot reproduce. It works with find \( -path
Yes, it turned out to be a problem with my syntax, I had to add a -not in there. I’ll delete my comments.
I’ve spent a couple of hours trying to find the answer to the same question, and I also wanted to understand what I was doing instead of just copying someones statement. This is definitely the best explanation I found. Thanks a lot.
«./tmp» and «/.scripts» only work if they’re a direct child of the current directory. Use «*/tmp» and «*/scripts» to make it also works if they’re in subdirectories (recursive).
Here is one way you could do it.
find . -type f -name "*_peaks.bed" | egrep -v "^(./tmp/|./scripts/)" This has the merit of working with any version of find , rather than only with GNU find . However, the question is tagged Linux so that is not critical.
for me, this solution didn’t worked on a command exec with find, don’t really know why, so my solution is
find . -type f -path "./a/*" -prune -o -path "./b/*" -prune -o -exec gzip -f -v <> \; Explanation: same as sampson-chen one with the additions of
-prune — ignore the proceding path of .
-o — Then if no match print the results, (prune the directories and print the remaining results)
18:12 $ mkdir a b c d e 18:13 $ touch a/1 b/2 c/3 d/4 e/5 e/a e/b 18:13 $ find . -type f -path "./a/*" -prune -o -path "./b/*" -prune -o -exec gzip -f -v <> \; gzip: . is a directory -- ignored gzip: ./a is a directory -- ignored gzip: ./b is a directory -- ignored gzip: ./c is a directory -- ignored ./c/3: 0.0% -- replaced with ./c/3.gz gzip: ./d is a directory -- ignored ./d/4: 0.0% -- replaced with ./d/4.gz gzip: ./e is a directory -- ignored ./e/5: 0.0% -- replaced with ./e/5.gz ./e/a: 0.0% -- replaced with ./e/a.gz ./e/b: 0.0% -- replaced with ./e/b.gz The accepted answer didn’t work, but this works. Using prune, find . -path ./scripts -prune -name ‘*_peaks.bed’ -type f . Not sure how to exclude multiple directories. This also lists the top level excluded directory even though type is specified. Excluding via Grep seems more straightforward unless you want to use prune to speed up the find operation.
I had trouble excluding multiple directories, too, but comments above gave me an answer that worked. I use multiple instances of ‘-not -path’ and in each path expression I include the full prefix as used in the first parameter to ‘find’ and end each with an asterisk (and escape any dots).
You can try below:
find ./ ! \( -path ./tmp -prune \) ! \( -path ./scripts -prune \) -type f -name '*_peaks.bed' On an old question like that (4 years!) you want to explain why this new answer is better or different, not just «dump» code.
With these explanations you meet your objective and many others. Just join each part as you want to do.
find ./\ -iname "some_arg" -type f\ # File(s) that you want to find at any hierarchical level. ! -iname "some_arg" -type f\ # File(s) NOT to be found on any hirearchic level (exclude). ! -path "./file_name"\ # File(s) NOT to be found at this hirearchic level (exclude). ! -path "./folder_name/*"\ # Folder(s) NOT to be found on this Hirearchic level (exclude). -exec grep -IiFl 'text_content' -- <> \; # Text search in the content of the found file(s) being case insensitive ("-i") and excluding binaries ("-I"). find ./\ -iname "*" -type f\ ! -iname "*pyc" -type f\ ! -path "./.gitignore"\ ! -path "./build/*"\ ! -path "./__pycache__/*"\ ! -path "./.vscode/*"\ ! -path "./.git/*"\ -exec grep -IiFl 'title="Brazil - Country of the Future",' -- <> \; [Ref(s).: https://unix.stackexchange.com/q/73938/61742 ]
You can use the commands above together with your favorite editor and analyze the contents of the files found, for example.
vim -p $(find ./\ -iname "*" -type f\ ! -iname "*pyc" -type f\ ! -path "./.gitignore"\ ! -path "./build/*"\ ! -path "./__pycache__/*"\ ! -path "./.vscode/*"\ ! -path "./.git/*"\ -exec grep -IiFl 'title="Brazil - Country of the Future",' -- <> \;) Ignore list of files in Find command Linux/ Unix [closed]
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
- This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
- We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
I would like to ignore list of files from the find command :
find . \( -name file1-o -name file2 \) In the above format, I have to give files individually. Can I put them into an array instead??
4 Answers 4
To answer your question, no, you can’t put the files to ignore into an array and expect to have find know about them. An array is an artifact of your shell (bash I assume), and the find tool is a separate binary, outside your shell.
That said, you can use an array to generate options for find.
#!/usr/bin/env bash a=(file1 file2 file3) declare -a fopt=() for f in "$"; do if [ "$" -eq 0 ]; then fopt+=("-name '$f'") else fopt+=("-o -name '$f'") fi done echo "find . -not ( $ )" There’s no doubt a more elegant way to handle the exclusion of -o from the first file found that will point out, but I haven’t had coffee yet this morning.
Exclude hidden files when searching with Unix/Linux find?
Aside: the reason there isn’t some special support for this task is that the only thing special about files named with a leading ‘.’ is that there are not listed by ls unless specifically requested: they are completely ordinary files in every respect, its just that ls lets you ignore them by default.
8 Answers 8
find . \( ! -regex '.*/\..*' \) -type f -name "whatever" @grawity I just found that, I don’t know entirely how it works. Would yours not only hide hidden files, but hidden directories and all their sub-content and hidden files in subfolders?
@grawity Ya, I guess I made an assumption about what the OP wanted. Your -name solution is probably the closest to what they were asking for.
@grawity&Dan: Isn’t it ( !-path ‘^.*’ ) ?? your solutions will ignore any file that has a ‘.’ anywhere in the file name like a.exe, b.out etc.
It seems negation glob pattern is not well known. So you can use:
note that this will still find non-hidden files in hidden directories, which may or may not be what you want.
This doesn’t answer your question, but for the task of finding non-hidden files I like to let find find all the files then filter with grep.
Similar to your approach but perhaps a bit simpler.
Try the following find usage:
Which would ignore all the hidden files (files and directories starting with a dot).
fd
Use fd , a simple, much faster and user-friendly alternative to find . By default, it:
- Ignores hidden directories and files, by default.
- Ignores patterns from your .gitignore , by default.
If you aims is to find and grep , ripgrep does exclude hidden files by default, e.g.
—files Print each file that would be searched without actually performing the search.
I wrote a script called findnh which I believe handles certain edge cases better than the answers to this question that I’ve been able to find on the web.
#!/bin/bash declare -a paths while [ $# -ne 0 ]; do case "$1" in -*) break ;; esac paths+=("$1") shift done find "$" \( -name . -o -name .. -o \! \( -name '.*' -prune \) \) "$@" For example, you can find non-hidden files and directories inside of an explicitly-specified hidden directory with a command like findnh ~/.hiddendir/ , which will show ~/.hiddendir/file but not ~/.hiddendir/.superhiddenfile .