Linux exclude files from copy

BASH copy all files except one

I would like to copy all files out of a dir except for one named Default.png. It seems that there are a number of ways to do this. What seems the most effective to you?

Why do you need it to skip that file, as opposed to just deleting it after copying it? Does it exist in the target directory already?

@LasseV.Karlsen: Or you could want to save the time of copying it, if it’s a large file. I’m interested in this but excluding a directory rather than a file.

9 Answers 9

If copying to a folder nested in the current folder (called example in the case below) you need to omit that directory also:

cp -r !(Default.png|example) /example 

It seems that OS X needs to use shopt -s extglob as described by @BarryKelly. With that, it works perfectly.

Years on Bash and didn’t know about !() . Beautiful! For those that —like me— feel it is time to study/review bash, here are the relevant links related to this question/answer: shopt/extglob and the pattern.

rsync has been my cp/scp replacement for a long time:

rsync -av from/ to/ --exclude=Default.png -a, --archive archive mode; equals -rlptgoD (no -H,-A,-X) -v, --verbose increase verbosity 

multiple —exclude= arguments are supported. And don’t forget the -r arg if you’re syncing directories

this»rsync -aP —exclude=backup /root/jenkins_api/* /root/jenkins_api/backup» does not work when there is no files in /root/jenkins_api/**.Is there a workaround to skip when no files are found ?

Simple, if src/ only contains files:

find src/ ! -name Default.png -exec cp -t dest/ <> + 

If src/ has sub-directories, this omits them, but does copy files inside of them:

find src/ -type f ! -name Default.png -exec cp -t dest/ <> + 

If src/ has sub-directories, this does not recurse into them:

find src/ -type f -maxdepth 1 ! -name Default.png -exec cp -t dest/ <> + 

@Max \; executes the command once per file. + runs the command once and passes all of the file names to it at once (subject to the command line length limit). + is a bit more efficient in general.

cp `ls | grep -v Default.png` destdir 

@arthur.sw For one thing, it doesn’t account for files with spaces in it. As with many things with the shell, there’s a bunch of edge cases that fail. 95%+ of the time, this stuff just works. Just need to be conscious of the caveats. Spaces in file names jump working with the shell from trivial to painful.

Another reason is that since there are no anchors in the regex and the dot isn’t escaped it would also match a file named myDefaultXpng.OLD for example.

cp srcdir/* destdir/ ; rm destdir/Default.png 

unless the files are big. Otherwise use e.g.

find srcdir -type f/ |grep -v Default.png$ |xargs -ILIST cp LIST destdir/ 

The first command is not what the OP asked for. If Default.png exists in the two directories, it will replace the one in destdir by the one in srcdir , then delete the copied Default.png . Instead, the OP wants to keep the Default.png that already exists in destdir .

Jan, 2022 Update:

This is the easiest way(It’s not complicated).

First, make «temp» folder:

Second, copy all files from your original folder to «temp» folder:

«-R» flag can copy exactly all files including «Symbolic Links»

Third, remove «Default.png» from «temp» folder:

Finally, copy all files from «temp» folder to your destination folder:

cp -R temp/. destinationFolder/ 

In addition, this is the shortest way without «temp» folder:

cp -R originalFolder/!(Default.png) destinationFolder/ 

Below script worked for me

cp -r `ls -A | grep -v 'skip folder/file name'` destination 

# chattr +i /files_to_exclude # cp source destination # chattr -i /files_to_exclude 

use the shell’s expansion parameter with regex

Everything will be copied except for the not_to_copy_file

— if something is wrong with this. please Specify !

Welcome to SO. Unfortunately your answer is not correct. The bracket expresssion ( [. ] ) contains a set of characters to match, while a leading ^ will cause a match of the complement of the listed characters. In the following example, neither file will be listed: touch not_to_copy_file to_copy_file ; ls [^not_to_copy_file]* because all filenames starting with any of the following characters will be excluded: _cefilnopty .

Источник

How to Use rsync to Exclude Files and Directories in Data Transfer

Rsync is a useful Linux command-line tool that syncs and copies files and directories. You can use the tool to synchronize data locally between directories and drives, or between two remote machines.

The basic rsync commands in Linux sync everything from a location you specify. In many backup scenarios, you may want to exclude specific files, directories, or file types.

Follow the examples in this guide to learn how to exclude files or directories with rsync. We will list the most common exclude use-cases to cover your day-to-day rsync usage.

• User with sudo or root privileges
• Access to a command line/terminal window
• Rsync installed on your system

How rsync Exclude Option Works

The —exclude option with the rsync command uses relative paths to the source directory. Append the —exclude option to the rsync command, followed by the relative path to a directory or file(s).

The basic syntax for the rsync exclude option looks like this:

rsync [OPTIONS] --exclude 'file_or_directory' source/ destination/

Replace source/ with the directory name you want to use as a source for data transfer to another location.

Replace destination/ with the directory name rsync will use as the target location for your data. If the directory does not exist, rsync creates one for you and transfers the files to that directory.

To back up files to a remote location, follow our other guide to learn how to copy or transfer files with rsync over SSH.

Note: The trailing slash (/) on the source directory modifies the behavior of the rsync command.

• If you do not use a trailing slash, the source directory is copied to the destination directory, and then the contents of the directory.
• When you do use the trailing slash, rsync only copies the content of the source without creating an additional directory level.

Exclude a Specific File

In the rsync exclude examples below, we will use the -a (archive) and -v (verbose) options ( -av ). The -a option syncs directories recursively while keeping permissions, symbolic links, ownership, and group settings. The -v flag is optional and prints the progress and status of the rsync command.

To exclude a file while transferring the contents of a folder with rsync, specify the file and the relative path.

rsync -av --exclude 'testfile1.txt' sourcedir/ destinationdir/

This command allows you to copy the files to destinationdir from sourcedir but exclude testfile1.txt.

As you can see in the output, testfile1.txt is not on the list.

Note: If you want to test all the examples in the guide, please remove the destination directory before trying a new example.

To do so, run this command:

sudo rm -rf destinationdir

Exclude a Specific Directory

Now that you know how to use the rsync —exclude file command, use the tool in the same fashion to exclude a directory.

Specify a directory you want to exclude (instead of a file name):

rsync -av --exclude 'dir1' sourcedir/ destinationdir/

This command copied the contents of sourcedir into destinationdir and excluded dir1, as seen in the output.

Exclude Files or Directories Based on a Pattern

Use an asterisk * (wildcard) when defining a file or directory name to exclude everything that matches the pattern.

To exclude files that start with test, run this command:

rsync -av --exclude 'test*' sourcedir/ destinationdir/

Every file and directory that matches this pattern will be excluded from the transfer.

You can also use the wildcard in a similar matter to exclude all directories that end with a specific pattern.

Run this command to exclude all directories that end with number 3:

rsync -av --exclude '*3' sourcedir/ destinationdir/

You can use an asterisk before and after a pattern to additionally refine the —exclude criterion.

Exclude a Specific File Type

The rsync tool allows you to exclude certain file types when synchronizing data. Use an asterisk * followed by the extension of the file type you want to exclude.

For example, you may want to back up a directory that contains many .iso files that you do not need to back up.

To exclude a specific file type, in this case .iso, run this command:

rsync -av --exclude '*.iso' sourcedir/ destinationdir/

The output shows that rsync did not transfer .iso files. We added the contents of the source directory in the image for comparison.

Exclude Files by Size

During file transfers, you can specify the minimum or the maximum size of files you want to exclude. If your source directory contains many large files that you do not want to back up, use the —max-size=size_in_mb_or_gb option. Replace the size_in_mb_or_gb with the desired size.

For example, to exclude all files larger than 500MB, run this command:

rsync -av --max-size=500m sourcedir/ destinationdir/

On the other hand, to exclude files smaller than a specific size, use the —min-size=size_in_mb_or_gb option.

For example, you want to transfer a directory with images, but there are many thumbnail files. If all your images are larger than 1MB, then you can exclude all files smaller than that size.

To do so, run this command:

rsync -av --min-size=1m sourcedir/ destinationdir/

Do not use the —exclude rsync option with when defining the minimum or maximum file size.

Exclude Multiple Files or Directories

Add multiple —exclude options to exclude multiple files or directories. You can combine any rsync —exclude folder and rsync —exclude file(s) commands to transfer only the data you need. Any of the commands we previously talked about can be used in one line.

The following command will exclude all files with the .txt extension, as well as the dir3 and dir4 directories:

rsync -av --exclude '*.txt' --exclude 'dir3' --exclude 'dir4' sourcedir/ destinationdir/

You can add as many —exclude entries as you need. To keep the command tidy, you can specify files and directories in the —exclude option using curly brackets. Separate the patterns using a comma.

For example, we can shorten the above command in this fashion:

rsync -av --exclude= sourcedir/ destinationdir/

The output shows that the listed files and directories are excluded from the transfer.

Exclude Files and Directories from a List

When you need to exclude a large number of different files and directories, you can use the rsync —exclude-from flag. To do so, create a text file with the name of the files and directories you want to exclude. Then, pass the name of the file to the —exlude-from option.

The command looks like this:

rsync -av --exclude-from= sourcedir/ destinationdir/

The rsync tool skips all files and directories you list in the file. You can add any pattern we used in this guide.

The example file we used in the above command contains the following patterns:

This guide showed you all rsync exclude examples you need when transferring data with this tool. You can combine the patterns to customize the command and exclude multiple files and directories with rsync.

Источник

How to copy some, but not all files?

So, you can use the * as a wild card for all files when using cp within context of a directory. Is there a way to copy all files except x file?

7 Answers 7

Rsync handles this nicely.

Example copy all: rsync -aP /folder1/* /folder/2

Example copy all with exclusion: rsync -aP —exclude=x /folder1/* /folder2/

On darwin/MacOS, use -rP instead of -aP if you want to recurse. -a is for archiving. Not sure if this changed or if it’s just different on MacOS.

rsync does have the option to make it recursive. Example: rsync —recursive -P —exclude=x /folder1/* /folder2/ . (Tested only on Ubuntu)

In bash you can use extglob :

 $ shopt -s extglob # to enable extglob $ cp !(b*) new_dir/ 

where !(b*) exclude all b* files.

You can later disable extglob with

Unfortunately I don’t. Seems like find is the only way in tcsh : find . -maxdepth 1 ! -name «exclude*» -exec cp -t destination <> \+

This isn’t a feature of cp , it’s a feature of your shell (it expands the * to mean all non-dot files), so the answer depends on which shell you’re using. For example, zsh supports this syntax:

Where ^x means «all files except x «

You can also combine selection and de-selection patterns, e.g. to copy all wav files except those containing xyz, you can use:

Could also be done in plain old (portable/compatible) bourne shell in a variety of ways with standard tools in a lot less elegant ways than using advanced shell globbing or commands with built-in-exclusion options.

If there are not too many files (and not with names including spaces and/or linebreaks), this could be a way:

cp `ls | egrep -v '^excludename$'` destdir/. 

Sure, bash and GNU tools are great and powerful, but they’re still not always available. If you intend to put it in a portable script, I would recommend find as in the comment by Rush.

Источник