How do I copy all files and directories, except certain ones over ssh?
Using the Linux command line, I use the scp command, to copy all the files and folders from a certain directory. However, I don’t like to consume bandwidth, for copying things I rarely change like my tiny_mce folder. What’s the trick to copy everything, but skip a short list of folders?
10 Answers 10
rsync works fine, and in most cases, uses SSH automatically as it’s transport protocol. It will compare files and only upload those that have changed — but you can also use an exclude list to specify files in the tree that shouldn’t be rsynced anyhow.
Yes, it does. Rsync 2.6.0 released — The default remote shell is now «ssh» unless you tell configure you want to make something else the default. samba.org/rsync
You could try rsync which only copies files that have changed, also works over ssh.
Using rsync —exclude is the more obvious choice here, but if you really want to just send a few files or folders and have something specific to exclude, you can use shell globing with scp . First make sure you have the right globing options set in your shell. For bash run shopt -s extglob and for zsh use setopt ksh_glob . Then something like this:
scp /path/to/folder/!(tiny_mce|other_folder|*.bak) user@host:target_path . would copy everything in the source folder except for things matching the given pattern. Obviously you can get creative with that part.
BASH copy all files except one
I would like to copy all files out of a dir except for one named Default.png. It seems that there are a number of ways to do this. What seems the most effective to you?
Why do you need it to skip that file, as opposed to just deleting it after copying it? Does it exist in the target directory already?
@LasseV.Karlsen: Or you could want to save the time of copying it, if it’s a large file. I’m interested in this but excluding a directory rather than a file.
9 Answers 9
If copying to a folder nested in the current folder (called example in the case below) you need to omit that directory also:
cp -r !(Default.png|example) /example It seems that OS X needs to use shopt -s extglob as described by @BarryKelly. With that, it works perfectly.
Years on Bash and didn’t know about !() . Beautiful! For those that —like me— feel it is time to study/review bash, here are the relevant links related to this question/answer: shopt/extglob and the pattern.
rsync has been my cp/scp replacement for a long time:
rsync -av from/ to/ --exclude=Default.png -a, --archive archive mode; equals -rlptgoD (no -H,-A,-X) -v, --verbose increase verbosity multiple —exclude= arguments are supported. And don’t forget the -r arg if you’re syncing directories
this»rsync -aP —exclude=backup /root/jenkins_api/* /root/jenkins_api/backup» does not work when there is no files in /root/jenkins_api/**.Is there a workaround to skip when no files are found ?
Simple, if src/ only contains files:
find src/ ! -name Default.png -exec cp -t dest/ <> + If src/ has sub-directories, this omits them, but does copy files inside of them:
find src/ -type f ! -name Default.png -exec cp -t dest/ <> + If src/ has sub-directories, this does not recurse into them:
find src/ -type f -maxdepth 1 ! -name Default.png -exec cp -t dest/ <> + @Max \; executes the command once per file. + runs the command once and passes all of the file names to it at once (subject to the command line length limit). + is a bit more efficient in general.
cp `ls | grep -v Default.png` destdir @arthur.sw For one thing, it doesn’t account for files with spaces in it. As with many things with the shell, there’s a bunch of edge cases that fail. 95%+ of the time, this stuff just works. Just need to be conscious of the caveats. Spaces in file names jump working with the shell from trivial to painful.
Another reason is that since there are no anchors in the regex and the dot isn’t escaped it would also match a file named myDefaultXpng.OLD for example.
cp srcdir/* destdir/ ; rm destdir/Default.png unless the files are big. Otherwise use e.g.
find srcdir -type f/ |grep -v Default.png$ |xargs -ILIST cp LIST destdir/ The first command is not what the OP asked for. If Default.png exists in the two directories, it will replace the one in destdir by the one in srcdir , then delete the copied Default.png . Instead, the OP wants to keep the Default.png that already exists in destdir .
Jan, 2022 Update:
This is the easiest way(It’s not complicated).
First, make «temp» folder:
Second, copy all files from your original folder to «temp» folder:
«-R» flag can copy exactly all files including «Symbolic Links»
Third, remove «Default.png» from «temp» folder:
Finally, copy all files from «temp» folder to your destination folder:
cp -R temp/. destinationFolder/ In addition, this is the shortest way without «temp» folder:
cp -R originalFolder/!(Default.png) destinationFolder/ Below script worked for me
cp -r `ls -A | grep -v 'skip folder/file name'` destination # chattr +i /files_to_exclude # cp source destination # chattr -i /files_to_exclude use the shell’s expansion parameter with regex
Everything will be copied except for the not_to_copy_file
— if something is wrong with this. please Specify !
Welcome to SO. Unfortunately your answer is not correct. The bracket expresssion ( [. ] ) contains a set of characters to match, while a leading ^ will cause a match of the complement of the listed characters. In the following example, neither file will be listed: touch not_to_copy_file to_copy_file ; ls [^not_to_copy_file]* because all filenames starting with any of the following characters will be excluded: _cefilnopty .
How to copy some, but not all files?
So, you can use the * as a wild card for all files when using cp within context of a directory. Is there a way to copy all files except x file?
7 Answers 7
Rsync handles this nicely.
Example copy all: rsync -aP /folder1/* /folder/2
Example copy all with exclusion: rsync -aP —exclude=x /folder1/* /folder2/
On darwin/MacOS, use -rP instead of -aP if you want to recurse. -a is for archiving. Not sure if this changed or if it’s just different on MacOS.
rsync does have the option to make it recursive. Example: rsync —recursive -P —exclude=x /folder1/* /folder2/ . (Tested only on Ubuntu)
In bash you can use extglob :
$ shopt -s extglob # to enable extglob $ cp !(b*) new_dir/ where !(b*) exclude all b* files.
You can later disable extglob with
Unfortunately I don’t. Seems like find is the only way in tcsh : find . -maxdepth 1 ! -name «exclude*» -exec cp -t destination <> \+
This isn’t a feature of cp , it’s a feature of your shell (it expands the * to mean all non-dot files), so the answer depends on which shell you’re using. For example, zsh supports this syntax:
Where ^x means «all files except x «
You can also combine selection and de-selection patterns, e.g. to copy all wav files except those containing xyz, you can use:
Could also be done in plain old (portable/compatible) bourne shell in a variety of ways with standard tools in a lot less elegant ways than using advanced shell globbing or commands with built-in-exclusion options.
If there are not too many files (and not with names including spaces and/or linebreaks), this could be a way:
cp `ls | egrep -v '^excludename$'` destdir/. Sure, bash and GNU tools are great and powerful, but they’re still not always available. If you intend to put it in a portable script, I would recommend find as in the comment by Rush.
Move all files except one
where I move old stuff to new stuff -folder except Tux.png . !-sign represents a negation. Is there some tool for the job?
14 Answers 14
If you use bash and have the extglob shell option set (which is usually the case):
mv ~/Linux/Old/!(Tux.png) ~/Linux/New/ I got something wrong when I tested the command for many items: mv ~/Linux/Old/!(Tux.png Tux1.png Tux2.png) ~/Linux/New/ It empties the whole Old -directory. What is wrong?
@Juliano Very cool command! Are there more mathematical operations like OR and XOR? I assume a pipe is for AND.
For ZSH user, instead of using shopt (which will give command not found error), add this to your .zshrc : setopt extended_glob then the syntax for glob will also change accordingly. Thus use mv ~/path/to/source^(exception) ~/path/to/target/folder should do
Put the following to your .bashrc
It extends regexes. You can then move all files except one by
Exceptions in relation to other commands
Note that, in copying directories, the forward-flash cannot be used in the name as noticed in the thread Why extglob except breaking except condition?:
cp -r !(Backups.backupdb) /home/masi/Documents/ so Backups.backupdb/ is wrong here before the negation and I would not use it neither in moving directories because of the risk of using wrongly then globs with other commands and possible other exceptions.
This is actually the correct answer. Mine was flat out wrong; luckily I had used it in a folder with only few files. Here is more information about the method Masi shows: wiki.bash-hackers.org/syntax/pattern Go to «Extended pattern language» and you will find more info on this. Thanks to @paul-whittaker for pointing at the issue.
This is an old answer but it’s still valid. I’m using Ubuntu 18 and extglob seems to be enabled by default (I didn’t shopt it). I used the following command to move all files in the current directory into an archive that is also within this directory, NOT including any other archive folders: mv !(arc*) archive_190629b
I would go with the traditional find & xargs way:
find ~/Linux/Old -maxdepth 1 -mindepth 1 -not -name Tux.png -print0 | xargs -0 mv -t ~/Linux/New -maxdepth 1 makes it not search recursively. If you only care about files, you can say -type f . -mindepth 1 makes it not include the ~/Linux/Old path itself into the result. Works with any filenames, including with those that contain embedded newlines.
One comment notes that the mv -t option is a probably GNU extension. For systems that don’t have it
find ~/Linux/Old -maxdepth 1 -mindepth 1 -not -name Tux.png \ -exec mv '<>' ~/Linux/New \;