Linux rsync skipping directory

rsync prints «skipping non-regular file» for what appears to be a regular directory

I back up my files using rsync. Right after a sync, I ran it expecting to see nothing, but instead it looked like it was skipping directories. I’ve (obviously) changed names, but I believe I’ve still captured all the information I could. What’s happening here?

$ ls -l /source/backup/myfiles drwxr-xr-x 2 me me 4096 2010-10-03 14:00 foo drwxr-xr-x 2 me me 4096 2011-08-03 23:49 bar drwxr-xr-x 2 me me 4096 2011-08-18 18:58 baz $ ls -l /destination/backup/myfiles drwxr-xr-x 2 me me 4096 2010-10-03 14:00 foo drwxr-xr-x 2 me me 4096 2011-08-03 23:49 bar drwxr-xr-x 2 me me 4096 2011-08-18 18:58 baz $ file /source/backup/myfiles/foo /source/backup/myfiles/foo/: directory 

$ rsync -rtvp /source/backup /destination sending incremental file list backup/myfiles skipping non-regular file "backup/myfiles/foo" skipping non-regular file "backup/myfiles/bar" 

$ echo 'hi' > /source/backup/myfiles/foo/test $ rsync -rtvp /source/backup /destination sending incremental file list backup/myfiles backup/myfiles/foo backup/myfiles/foo/test skipping non-regular file "backup/myfiles/foo" skipping non-regular file "backup/myfiles/bar" 

$ ls -l /source/backup/myfiles/foo -rw-r--r-- 1 me me 3126091 2010-06-15 22:22 IMGP1856.JPG -rw-r--r-- 1 me me 3473038 2010-06-15 22:30 P1010615.JPG -rw-r--r-- 1 me me 3 2011-08-24 13:53 test $ ls -l /destination/backup/myfiles/foo -rw-r--r-- 1 me me 3126091 2010-06-15 22:22 IMGP1856.JPG -rw-r--r-- 1 me me 3473038 2010-06-15 22:30 P1010615.JPG -rw-r--r-- 1 me me 3 2011-08-24 13:53 test 

$ rsync -rtvp /source/backup /destination sending incremental file list backup/myfiles skipping non-regular file "backup/myfiles/foo" skipping non-regular file "backup/myfiles/bar" 

Other notes: My actual directories «foo» and «bar» do have spaces, but no other strange characters. Other directories have spaces and have no problem. I ‘stat’-ed and saw no differences between the directories that don’t rsync and the ones that do. If you need more information, just ask.

Источник

Как заставить RSYNC копировать все файлы и папки?

Здравствуйте товарищи! Подскажите пожалуйста, как правильно настроить rsync.
Пытаюсь скопировать вот такой командой, но копируются только файлы. Пишет skipping directory test1
sudo rsync -zvh /home/user/documents/* /tmp/backups/

Иван Анатоличь, Если вам ответ помог — отмечайте решением, если не помог или не понравился — не отмечайте.
Тут никто никому не должен.
Вам человек вполне адекватно ответил, в справке все есть.

Иван Анатоличь, не используйте линукс. Не ваше это.
Поищите, как вашу задачу на винде решать. Может там гуишная приблуда есть какая то (

A trailing slash on the source changes this behavior to avoid creating an additional directory level at the destination.

Т.е. нужно убрать слэш в пути, чтобы создавались подкаталоги.
Там же написано, что, например, эти команды скопируют одно и то же:

rsync -av /src/foo /dest rsync -av /src/foo/ /dest/foo

Войдите, чтобы написать ответ

Как настроить vpn?

Источник

Why is Rsync Skipping Directories

The Rsync command is a popular utility for synchronizing files and directories between two remote systems.

Unlike other utilities, Rsync is highly recommended for transferring large amounts of data over a network. That’s because it can efficiently transfer only the portions of files that have changed rather than transferring the entire file each time.

Some of the popular Rsync features include

• Efficiency: As mentioned, Rsync can transfer only the parts of a file that have changed, which can significantly reduce the amount of data that needs to be transferred.
• Flexibility: Rsync has a wide range of options that allow users to customize its behavior to fit their specific needs. For example, users can specify the type of files to be transferred, exclude specific files or directories from being transferred, and control how Rsync handles file permissions and ownership.
• Versatility: Rsync can be used to transfer files over a network using various protocols, including ssh, rsh, and rsync itself. It can also be used to synchronize files between local directories.
• Wide availability: Rsync is available for different platforms, including Linux, macOS, Single Board devices (Raspberry Pi, etc.), and Windows.

Table of Contents

Rsync Command Skipping Directories – Error

If you have been working with the Rsync command for a while, you have likely encountered the error “skipping directory.” Rsync throws this error on the Terminal when it can’t sync/copy a specified directory. Take a look at the example below.

We have a “backup” directory that we wish to copy to our remote server – “server-one.” Unfortunately, when we executed the command below on our Terminal, we got the error “skipping directory.”

When we ran the ls command on the server, we saw that neither the directory nor the files inside the “backup” directory were synced/copied. So, why did Rsync skip this particular directory? What is the solution to such an error?

Solutions

There are several possible reasons why the Rsync command might skip directories. This post will look at each scenario and recommend a solution.

Not syncing/ copying a Directory Recursively (-r or -a).

“Recursive” is a popular term you will always come across when working with files and directories on the Linux Terminal. But what exactly does it mean?

When you add the recursive option ( -r ) in the Rsync syntax, that means the command will copy the directory itself and all of the files and subdirectories inside it, all of the files and subdirectories inside those subdirectories, and so on. In other words, the Rsync command will traverse the directory tree recursively and copy everything it finds.

However, the Rsync command will skip that directory if you don’t include the -r parameter when copying a directory. Take a look at the example below.

After running the command above, we got the error shown in the image below.

This error occurred because we tried to copy a directory without the recursive ( -r ) parameter. The solution is to add the -r parameter in the Rsync command. See below.

When you look at the image below, you can see that we did not get any error, and the files inside the directory were copied successfully.

Tip: Note that the Rsync command also has a -a option, which is equivalent to -dpR , and which combines the -d , -p , and -R options. The -d option tells the Rsync command to preserve symbolic links, the -p option tells the cp command to preserve the file attributes of the original files (such as ownership and permissions), and the -R option is equivalent to the -r option and tells the Rsync command to copy directories and their contents recursively.

Using Filtering Options (—include, —exclude, and —filter).

The —include , —exclude , and —filter options allow you to specify patterns for including or excluding files and directories from the transfer. If a directory doesn’t match any of the specified patterns, it will be skipped. Let’s have a more in-depth look using the —exclude option.

The —exclude option in the Rsync command allows you to specify patterns or files that should be excluded from the synchronization process. This can be useful if you want to exclude certain types of files or directories from being transferred or if you want to exclude specific files or directories by name.

To use the —exclude option, you will need to specify the pattern or name of the file or directory that you want to exclude. For example:

rsync -av --exclude 'test-directory' /source/ /destination/

This command will synchronize the contents of the /source/ directory with the /destination/ directory, but it will the test-directory from the synchronization process. Take a look at the scenario below.

We want to copy the files and directories inside the “Test-Directory” to our remote server. However, we want to exclude the “dir-1” directory. We will use the command below.

After running the command above, Rsync will copy all the files and folders except the “dir-1” directory. See the image below.

Permission issues

Rsync will skip a directory if it doesn’t have the necessary permissions to access it. By default, Rsync will not copy files or directories for which it does not have read permissions.

If you want Rsync to copy these files or directories, you can use the —ignore-perms option. This will cause Rsync to ignore permissions and copy the files or directories regardless of whether it has the necessary permissions.

However, it is generally not a good idea to use this option unless you are sure you want to ignore permissions and understand the implications of doing so.

Using an Outdated File List with the —delete Option

The —delete option tells Rsync to delete any files or directories at the destination that are not present at the source. This can be useful for ensuring that the destination is an exact copy of the source, but it can also lead to unexpected results if the file list used by Rsync is outdated.

The —delete option can cause Rsync to skip a directory if it is not present in the source directory and the file list used by Rsync is outdated.

For example, if you run Rsync with the —delete option, and the source directory contains a file or directory that has since been deleted, Rsync will delete the corresponding file or directory at the destination as well.

Similarly, if you run Rsync with the —delete option, and the file list used by Rsync is outdated such that it includes a directory that no longer exists in the source, Rsync will skip that directory and not delete it at the destination.

In general, it is a good idea to be cautious when using the —delete option, as it can lead to data loss if used improperly. It is always a good idea to make sure that you have a backup of your data before using this option, just in case.

Symbolic Links

A symbolic link, also known as a symlink or soft link, is a special type of file that points to another file or directory on a computer system. When you access a symbolic link, you effectively access the file or directory it points to.

The —copy-links option tells Rsync to follow symbolic links and transfer the files they point to rather than transferring the symbolic links themselves. This means that if you have a symbolic link pointing to a file, Rsync will transfer the actual file rather than the symbolic link.

The —copy-unsafe-links option is similar to —copy-links , but it also allows Rsync to follow symbolic links that point to files or directories outside the source and destination directories. This can be useful if you want to transfer files that are linked to by symbolic links in different locations on the file system.

However, it also raises one major problem. If a symbolic link points to a directory that doesn’t exist, Rsync will skip it. This means that Rsync will not follow the link or transfer any files that are linked to by the symbolic link.

Conclusion

This post has given you a comprehensive guide on why the Rsync command is skipping certain directories.

If you are experiencing this issue and are not sure what is causing it, you can try adding the —verbose or -v option to your Rsync command to see a detailed log of what Rsync is doing. That will greatly help to identify the cause of the problem.

Do you have any comments or queries regarding this post? Please don’t hesitate to let us know in the comments.

Источник