How to pipe the results of ‘find’ to mv in Linux
How do I pipe the results of a ‘find’ (in Linux) to be moved to a different directory? This is what I have so far.
find ./ -name '*article*' | mv ../backup but its not yet right (I get an error missing file argument, because I didn’t specify a file, because I was trying to get it from the pipe)
6 Answers 6
find ./ -name '*article*' -exec mv <> ../backup \; find ./ -name '*article*' | xargs -I '<>' mv <> ../backup when I ran your second command, it says xargs: invalid option — I. What is wrong here? I see that it is a valid option on the man page.
no. If it helps, I am in a git Bash on a windows machine. It seems to be running linux commands, but I guess it doesn’t have the full library?
@user1015214 — Probably git bash is limited. Since you are on windows machine, I’d suggest using Move-Item Cmdlet in Powershell . You can do Move-Item *article* \full\path\to\backup
xargs is commonly used for this, and mv on Linux has a -t option to facilitate that.
find ./ -name '*article*' | xargs mv -t ../backup If your find supports -exec . + instead of -exec . \; you could equivalently do
find ./ -name '*article*' -exec mv -t ../backup <> + The -t option is a GNU extension, so it is not portable to systems which do not have GNU coreutils (though every proper Linux I have seen has that, with the possible exception of Busybox). For complete POSIX portability, it’s of course possible to roll your own replacement, maybe something like
find ./ -name '*article*' -exec sh -c 'mv "$@" "$0"' ../backup <> + where we shamelessly abuse the convenient fact that the first argument after sh -c ‘commands’ ends up as the «script name» parameter in $0 so that we don’t even need to shift it.
When I tried you first suggestion, it said ‘invalid option — t’. When I tried your second one, it said -exec command not found, so I guess its not supported.
Sounds like you are not on Linux after all then, as this has been supported by the GNU versions of these tools for a long time. I can’t find (sic) a find which doesn’t support -exec in some form — maybe you mistyped the command?
@exclowd The find -exec variant is specifically robust against all kinds of problems with unusual file names. The xargs variant is slightly less so, though the linked FAQ has a more elaborate variation which requires GNU find (so basically Linux only). In brief, find . -print0 | xargs -r0 mv -t ../backup
I found this really useful having thousands of files in one folder:
ls -U | head -10000 | egrep '\.png$' | xargs -I '<>' mv <> ./png To move all pngs in first 10000 files to subfolder png
The regex has no egrep (aka grep -E more modernly) syntax, and you can factor out the head : grep -m 10000 ‘\.png$’ , More generally you should avoid ls in scripts,
Perhaps then printf ‘%s\0’ *.png | xargs -r0 mv -t png/ which avoids the various issues with ls , and printf ‘%s\0’ *.png | grep -z -m 10000 ^ | xargs -r0 mv -t png/ to coincidentally use grep -z -m items as a replacement for head which groks null-terminated input. The link I posted earlier explains these things in much more detail. I’m divided about improving an answer which advocates ls ; but feel free to update as you see fit, and include anything or everything from these comments. Incidentally, xargs -0 is GNU only (should work on Linux, but isn’t properly portable).
How to use a pipe with Linux find command?
Linux find statement is one of the most widely used statements that allows us to walk a file hierarchy. It is used to mostly find a specific file or directories and we can also append different other Linux statements or flags along with it to enhance or do a complex operation.
Let’s explore an example of a find statement to understand it better.
In the Linux code shown below, I am trying to search for a file inside my Downloads folder, and for that I am making use of the find statement
Output
Notice that if the find command is able to locate the file then it will simply print the name of the file, if not then it will not return anything the terminal process will terminate.
Now we know how the find statement works, let’s explore the case where we want to use the find command along with a pipe.
A pipe in Linux is just a vertical bar on your keyboard. It is used to consider the command that is on the left side of it as an input to the command on the right side of it.
Now that we know about both the find command the pipe in Linux, let’s consider an example where we will make use of both of these in a Linux command.
Command
find . -name '*.txt' | xargs cat
In the above command, just before the pipe, I am considering all the files that has a name that ends with a .txt extension and then after the pipe, I am simply printing those with the help of the cat command.
Output
immukul@192 directory1 % find . -name '*.txt' | xargs cat this is a test file and it is used for testing and is not available for anything else so please stop asking, lionel messi orange orange blabla blabla foofoo here is the text to keep between the 2 patterns bar blabla blabla
How do I include a pipe | in my linux find -exec command?
What the OP was trying to accomplish can be met with the suggestions above, but this is the one which actually answers the question asked. There are reasons to do it this way — exec is a lot more powerful than just operating on the files returned by find, especially when combined with test. For instance: find geda-gaf/ -type d -exec bash -c ‘DIR=<>; [[ $(find $DIR -maxdepth 1 |xargs grep -i spice |wc -l) -ge 5 ]] && echo $DIR’ \; Will return all directories in the search path which contain more than 5 lines total among all the files in that directory containing the word spice
Best answer. Grepping the whole output (as other answers suggest) isn’t the same as grep each file. Tip: instead of sh, you can use any other shell you want (I tried that with bash and it’s running ok).
Make sure to not overlook the -c option. Otherwise you will get a puzzling No such file or directory error message.
Example of finding files and renaming them with sed using regular expression find -type f -name ‘*.mdds’ -exec sh -c «echo <> | sed -e ‘s/_1\+//g’ | xargs mv <>» \;
@Roland try to double quote the brackets and single quotes on the outer scope -type f -exec sh -c ‘echo «<>«‘ | grep something\;
The job of interpreting the pipe symbol as an instruction to run multiple processes and pipe the output of one process into the input of another process is the responsibility of the shell (/bin/sh or equivalent).
In your example you can either choose to use your top level shell to perform the piping like so:
find -name 'file_*' -follow -type f -exec zcat <> \; | agrep -dEOE 'grep' In terms of efficiency this results costs one invocation of find, numerous invocations of zcat, and one invocation of agrep.
This would result in only a single agrep process being spawned which would process all the output produced by numerous invocations of zcat.
If you for some reason would like to invoke agrep multiple times, you can do:
find . -name 'file_*' -follow -type f \ -printf "zcat %p | agrep -dEOE 'grep'\n" | sh This constructs a list of commands using pipes to execute, then sends these to a new shell to actually be executed. (Omitting the final «| sh» is a nice way to debug or perform dry runs of command lines like this.)
In terms of efficiency this results costs one invocation of find, one invocation of sh, numerous invocations of zcat and numerous invocations of agrep.
The most efficient solution in terms of number of command invocations is the suggestion from Paul Tomblin:
find . -name "file_*" -follow -type f -print0 | xargs -0 zcat | agrep -dEOE 'grep' . which costs one invocation of find, one invocation of xargs, a few invocations of zcat and one invocation of agrep.
Another advantage of xargs would be, that you can speed it with modern multi core cpu even more up, by using the -P switch (-P 0).
Yes, the -P swich is indeed a nice way to speed up execution in general. Unfortunately, you run the risk of the output of parallel zcat processes being piped into agrep interleaved, which would affect the result. This effect can be demonstrated using: echo -e «1\n2» | xargs -P 0 -n 1 yes | uniq
A simpler and more general answer is at stackoverflow.com/a/21825690/42973: -exec sh -c «… | … » \; .
find . -name "file_*" -follow -type f -print0 | xargs -0 zcat | agrep -dEOE 'grep' Hoping to avoid -print and xargs for efficiency reasons. Maybe that’s really my problem: find cannot handle piped commands through -exec
This doesn’t work with files with spaces in their names; to fix, replace -print with -print0 and add the -0 option to xargs
@someguy — Wha? Avoiding xargs for efficiency reasons? Calling one instance of zcat, and passing it a list of multiple files, is far more efficient than exec-ing a new instance of it for each found file.
@Adam — I’ve made your suggested change. 99% of the time when I’m doing finds, it’s in my source code directories, and none of the files there have spaces so I don’t bother with print0. Now my documents directory, on the other hand, I remember the print0.
Would be a great answer with a little explanation about -print0 , xargs -0 , and filename with spaces.
You can also pipe to a while loop that can do multiple actions on the file which find locates. So here is one for looking in jar archives for a given java class file in folder with a large distro of jar files
find /usr/lib/eclipse/plugins -type f -name \*.jar | while read jar; do echo $jar; jar tf $jar | fgrep IObservableList ; done the key point being that the while loop contains multiple commands referencing the passed in file name separated by semicolon and these commands can include pipes. So in that example I echo the name of the matching file then list what is in the archive filtering for a given class name. The output looks like:
/usr/lib/eclipse/plugins/org.eclipse.core.contenttype.source_3.4.1.R35x_v20090826-0451.jar /usr/lib/eclipse/plugins/org.eclipse.core.databinding.observable_1.2.0.M20090902-0800.jar org/eclipse/core/databinding/observable/list/IObservableList.class /usr/lib/eclipse/plugins/org.eclipse.search.source_3.5.1.r351_v20090708-0800.jar /usr/lib/eclipse/plugins/org.eclipse.jdt.apt.core.source_3.3.202.R35x_v20091130-2300.jar /usr/lib/eclipse/plugins/org.eclipse.cvs.source_1.0.400.v201002111343.jar /usr/lib/eclipse/plugins/org.eclipse.help.appserver_3.1.400.v20090429_1800.jar
in my bash shell (xubuntu10.04/xfce) it really does make the matched classname bold as the fgrep highlights the matched string; this makes it really easy to scan down the list of hundreds of jar files that were searched and easily see any matches.
on windows you can do the same thing with:
for /R %j in (*.jar) do @echo %j & @jar tf %j | findstr IObservableList note that in that on windows the command separator is ‘&’ not ‘;’ and that the ‘@’ suppresses the echo of the command to give a tidy output just like the linux find output above; although findstr is not make the matched string bold so you have to look a bit closer at the output to see the matched class name. It turns out that the windows ‘for’ command knows quite a few tricks such as looping through text files.