Copy specific file type keeping the folder structure
I have a folder structure with a bunch of *.csv files scattered across the folders. Now I want to copy all *.csv files to another destination keeping the folder structure. It works by doing:
cp --parents *.csv /target cp --parents */*.csv" /target cp --parents */*/*.csv /target cp --parents */*/*/*.csv /target . 7 Answers 7
find has a very handy -exec option:
find . -name '*.csv' -exec cp --parents \ /target \; Even though -execdir is safer than -exec , simply replacing one with the other does not preserve the folder structure as intended.
You could also use rsync for this.
$ rsync -a --prune-empty-dirs --include '*/' --include '*.csv' --exclude '*' source/ target/ If you want to keep empty directories from the source tree, skip the —prune-empty-dirs option:
$ rsync -a --include '*/' --include '*.csv' --exclude '*' source/ target/ If you do not want symlinks, modification dates, file permissions, owners etc. preserved, please replace -a with another combination of -rlptgoD . 😉
Also, the -R option can be added to copy the parent directory structure of source. (cf. my answer here.)
Unfortunately it seems that rsync is excruciating slow at finding matching files though. It takes like 100 more time than find. Is there anything that can be done about this?
You can use find and cpio in pass through mode
find . -name '*.csv' | cpio -pdm /target This will find all .csv files in the current directory and below and copy them to /target maintaining the directory structure rooted in . .
find /path/to/files -name '*.csv' | cpio -pdm /target it will find all of the file in /path/to/files and below and copy them to /target/path/to/files and below.
I tried all the answers top down up to this one, and this was the only one that worked on the first attempt
I found the GNU docs helpful: «In copy-pass mode, [requested by the -p option, cpio] reads the list of files to copy from the standard input; the directory into which it will copy them is given as a non-option argument.» . «-d Create leading directories where needed.» . «-m Retain previous file modification times when creating files.»
I want to do the same as the first line but copy all the contents of a folder and not only the csv files.
The cp command permits multiple source arguments:
CAVEAT: I’m using a recursive glob here; this is the globstar option in Bash 4+ and ksh , and is supported by default in zsh . Recursive globs do not match hidden files and folders, and the some implementations follow symlinks while others do not.
If your shell doesn’t support recursive globs, or if you’d prefer not to use them, you can do the following:
- *.csv */*.csv */*/*.csv */*/*/*.csv — this is of course very redundant and requires knowing how deep your directory structure is.
- $(find . -name ‘*.csv’) — This will match hidden files and folders. find also supports specifying whether or not symlinks are followed, which may be useful.
this is exactly what i tried (the recursive glob) and it found some but not all? pretty weird. I got the same exact result when using the npm copyfiles script but if i use the find command it finds everything.
@Randyaa I’ll need some more details on which files, exactly, weren’t found in order to help you. You may find the discussion here and continued here about the precise behavior of the recursive glob useful.
turns out recursive glob wasn’t enabled for some reason. I’ve never run into this before but i corrected it with just an execution of shopt -s globstar immediately before my command and all is well. Thanks for the follow up!
find -name «*.csv» | xargs cp —parents -t /target
If you have file names with spaces, add options -print0 and -0 like suggested in one of the comments:
find -name «*.csv» -print0 | xargs -0 cp —parents -t /target
Best answer with most simple syntax. Works just fine and is easy to remember. In many cases find [things] | xargs [do stuff] is very powerfull.
@pasztorpisti, is there any downside to that? If not should the answer not simply be edited to include your suggestion.
-R, —relative
Use relative paths. This means that the full path names specified on the command line are sent to the server rather than just the last parts of the filenames. This is particularly useful when you want to send several different directories at the same time. For example, if you used this command:
rsync -av /foo/bar/baz.c remote:/tmp/ . this would create a file named baz.c in /tmp/ on the remote machine. If instead you used
rsync -avR /foo/bar/baz.c remote:/tmp/ then a file named /tmp/foo/bar/baz.c would be created on the remote machine, preserving its full path. These extra path elements are called «implied directories» (i.e. the «foo» and the «foo/bar» directories in the above example).
rsync -armR --include="*/" --include="*.csv" --exclude="*" /full/path/to/source/file(s) destination/ How to copy only files in a specified directory to another folder
I’m trying to copy only the files from one directory (not including folders or any files in its subfolders) to another location using cp /media/d/folder1/* /home/userA/folder2/ . It is copying the files alright but the problem is that there are a list of messages appear saying cp: omitting directory. for all the folders located in folder1 . Is there any other way to copy these folders without having this message appearing? Also another thing please, I’m asking the same thing if I wants to move (not copy), how this can be done? Thanks
If you are only annoyed by the messages you can redirect them by appending 2>/dev/null . (No need for complex commands and command pipes.)
3 Answers 3
find /media/d/folder1/ -maxdepth 1 -type f | xargs cp -t /home/userA/folder2 Part before the pipe character | finds the files in the given directory without attempting to find other files under any sub-directory of the given directory. Part after the pipe takes those files and copies them to the destination directory. You can change cp command with mv if you want to move files instead of copying.
I’d change this to find /media/d/folder1/ -maxdepth 1 -type f -print0 | xargs -0 cp -t /home/userA/folder2 ao that files with special characters (such as spaces) are also copied.
A safer approach (that can deal with file names with spaces, newlines and other odd characters) is to use find itself and its -exec action:
-exec command <> + This variant of the -exec action runs the specified command on the selected files, but the command line is built by appending each selected file name at the end; the total number of invoca‐ tions of the command will be much less than the number of matched files. The command line is built in much the same way that xargs builds its command lines. Only one instance of `<>' is allowed within the command. The command is executed in the starting directory. find /media/d/folder1/ -maxdepth 1 -type f -exec cp <> -t /home/userA/folder2 Note that this wil also copy hidden files.
How to copy some, but not all files?
So, you can use the * as a wild card for all files when using cp within context of a directory. Is there a way to copy all files except x file?
7 Answers 7
Rsync handles this nicely.
Example copy all: rsync -aP /folder1/* /folder/2
Example copy all with exclusion: rsync -aP —exclude=x /folder1/* /folder2/
On darwin/MacOS, use -rP instead of -aP if you want to recurse. -a is for archiving. Not sure if this changed or if it’s just different on MacOS.
rsync does have the option to make it recursive. Example: rsync —recursive -P —exclude=x /folder1/* /folder2/ . (Tested only on Ubuntu)
In bash you can use extglob :
$ shopt -s extglob # to enable extglob $ cp !(b*) new_dir/ where !(b*) exclude all b* files.
You can later disable extglob with
Unfortunately I don’t. Seems like find is the only way in tcsh : find . -maxdepth 1 ! -name «exclude*» -exec cp -t destination <> \+
This isn’t a feature of cp , it’s a feature of your shell (it expands the * to mean all non-dot files), so the answer depends on which shell you’re using. For example, zsh supports this syntax:
Where ^x means «all files except x «
You can also combine selection and de-selection patterns, e.g. to copy all wav files except those containing xyz, you can use:
Could also be done in plain old (portable/compatible) bourne shell in a variety of ways with standard tools in a lot less elegant ways than using advanced shell globbing or commands with built-in-exclusion options.
If there are not too many files (and not with names including spaces and/or linebreaks), this could be a way:
cp `ls | egrep -v '^excludename$'` destdir/. Sure, bash and GNU tools are great and powerful, but they’re still not always available. If you intend to put it in a portable script, I would recommend find as in the comment by Rush.
BASH copy all files except one
I would like to copy all files out of a dir except for one named Default.png. It seems that there are a number of ways to do this. What seems the most effective to you?
Why do you need it to skip that file, as opposed to just deleting it after copying it? Does it exist in the target directory already?
@LasseV.Karlsen: Or you could want to save the time of copying it, if it’s a large file. I’m interested in this but excluding a directory rather than a file.
9 Answers 9
If copying to a folder nested in the current folder (called example in the case below) you need to omit that directory also:
cp -r !(Default.png|example) /example It seems that OS X needs to use shopt -s extglob as described by @BarryKelly. With that, it works perfectly.
Years on Bash and didn’t know about !() . Beautiful! For those that —like me— feel it is time to study/review bash, here are the relevant links related to this question/answer: shopt/extglob and the pattern.
rsync has been my cp/scp replacement for a long time:
rsync -av from/ to/ --exclude=Default.png -a, --archive archive mode; equals -rlptgoD (no -H,-A,-X) -v, --verbose increase verbosity multiple —exclude= arguments are supported. And don’t forget the -r arg if you’re syncing directories
this»rsync -aP —exclude=backup /root/jenkins_api/* /root/jenkins_api/backup» does not work when there is no files in /root/jenkins_api/**.Is there a workaround to skip when no files are found ?
Simple, if src/ only contains files:
find src/ ! -name Default.png -exec cp -t dest/ <> + If src/ has sub-directories, this omits them, but does copy files inside of them:
find src/ -type f ! -name Default.png -exec cp -t dest/ <> + If src/ has sub-directories, this does not recurse into them:
find src/ -type f -maxdepth 1 ! -name Default.png -exec cp -t dest/ <> + @Max \; executes the command once per file. + runs the command once and passes all of the file names to it at once (subject to the command line length limit). + is a bit more efficient in general.
cp `ls | grep -v Default.png` destdir @arthur.sw For one thing, it doesn’t account for files with spaces in it. As with many things with the shell, there’s a bunch of edge cases that fail. 95%+ of the time, this stuff just works. Just need to be conscious of the caveats. Spaces in file names jump working with the shell from trivial to painful.
Another reason is that since there are no anchors in the regex and the dot isn’t escaped it would also match a file named myDefaultXpng.OLD for example.
cp srcdir/* destdir/ ; rm destdir/Default.png unless the files are big. Otherwise use e.g.
find srcdir -type f/ |grep -v Default.png$ |xargs -ILIST cp LIST destdir/ The first command is not what the OP asked for. If Default.png exists in the two directories, it will replace the one in destdir by the one in srcdir , then delete the copied Default.png . Instead, the OP wants to keep the Default.png that already exists in destdir .
Jan, 2022 Update:
This is the easiest way(It’s not complicated).
First, make «temp» folder:
Second, copy all files from your original folder to «temp» folder:
«-R» flag can copy exactly all files including «Symbolic Links»
Third, remove «Default.png» from «temp» folder:
Finally, copy all files from «temp» folder to your destination folder:
cp -R temp/. destinationFolder/ In addition, this is the shortest way without «temp» folder:
cp -R originalFolder/!(Default.png) destinationFolder/ Below script worked for me
cp -r `ls -A | grep -v 'skip folder/file name'` destination # chattr +i /files_to_exclude # cp source destination # chattr -i /files_to_exclude use the shell’s expansion parameter with regex
Everything will be copied except for the not_to_copy_file
— if something is wrong with this. please Specify !
Welcome to SO. Unfortunately your answer is not correct. The bracket expresssion ( [. ] ) contains a set of characters to match, while a leading ^ will cause a match of the complement of the listed characters. In the following example, neither file will be listed: touch not_to_copy_file to_copy_file ; ls [^not_to_copy_file]* because all filenames starting with any of the following characters will be excluded: _cefilnopty .