Linux copy file with progress

How can I make a progress bar while copying a directory with cp?

I suppose I could compare the number of files in the source directory to the number of files in the target directory as cp progresses, or perhaps do it with folder size instead? I tried to find examples, but all bash progress bars seem to be written for copying single files. I want to copy a bunch of files (or a directory, if the former is not possible).

11 Answers 11

You can also use rsync instead of cp like this:

rsync -Pa source destination

Which will give you a progress bar and estimated time of completion. Very handy.

Right, rsync —progress /path/to/origin /path/to/destination is awesome and is available on all systems.

@Allasso Tough to say because there’s so many options and situations which affect it. It can be slower for certain sets files, especially with a small CPU, because it’s analyzing/syncing not copying, and encrypting files (when over a network, iirc). For example see superuser.com/questions/109780/how-to-speed-up-rsync or superuser.com/questions/153176/…

To show a progress bar while doing a recursive copy of files & folders & subfolders (including links and file attributes), you can use gcp (easily installed in Ubuntu and Debian by running «sudo apt-get install gcp»):

Here is the typical output while copying a large folder of files:

Copying 1.33 GiB 73% |##################### | 230.19 M/s ETA: 00:00:07 

Notice that it shows just one progress bar for the whole operation, whereas if you want a single progress bar per file, you can use rsync :

rsync -ah --progress SRC DEST 

gcp is Python-based. PLEASE DO NOT USE IN PRODUCTION. I did not check, but it seems does not support hardlinks, special attributes and so on .

You may have a look at the tool vcp . Thats a simple copy tool with two progress bars: One for the current file, and one for overall.

Most distributions have a package for it.

I was hoping for something that doesn’t require compiling external tools. I just want to see my directory get copied. Is it really so difficult?

@Anonymouse added a second answer, maybe this would be an alternative for you. But thats my last idea 🙁

The members.iinet.net.au/~lynx/vcp link is broken. There seems to be a current fork at github.com/gdm85/curses-vcp with a last commit of May 29, 2016. I was able to compile it just fine on Fedora 23. An independent vcp with a similar concept is at github.com/lynix/vcp but I have not tried it.

Here another solution: Use the tool bar

You could invoke it like this:

#!/bin/bash filesize=$(du -sb $ | awk '< print $1 >') tar -cf - -C $ ./ | bar --size $ | tar -xf - -C $

You have to go the way over tar, and it will be inaccurate on small files. Also you must take care that the target directory exists. But it is a way.

Early on, I thought about tarring the folder before moving it, but thought it would lack too much in elegance. I was wrong. It works as expected and might actually be a better solution in some cases. Thanks!

My preferred option is Advanced Copy, as it uses the original cp source files.

$ wget http://ftp.gnu.org/gnu/coreutils/coreutils-8.21.tar.xz $ tar xvJf coreutils-8.21.tar.xz $ cd coreutils-8.21/ $ wget --no-check-certificate wget https://raw.githubusercontent.com/jarun/advcpmv/master/advcpmv-0.8-8.32.patch $ patch -p1 -i advcpmv-0.8-8.32.patch $ ./configure $ make 

The new programs are now located in src/cp and src/mv. You may choose to replace your existing commands:

$ sudo cp src/cp /usr/local/bin/cp $ sudo cp src/mv /usr/local/bin/mv 

Then you can use cp as usual, or specify -g to show the progress bar:

The link to the patch at zwicke.org is broken but it’s on github at github.com/atdt/advcpmv. Perhaps someone with more skills than I will update @elboletaire’s steps.

This should be selected answer. Better progress bar than rsync if copying several files. Better than gcp because reusing standard cp code from coreutils. Advantage of gcp is that it is distributed as a standard package.

Would be worth mentioning that you need to use -g option with this patched cp in order to get the progress bar.

A simple unix way is to go to the destination directory and do watch -n 5 du -s . Perhaps make it more pretty by showing as a bar . This can help in environments where you have just the standard unix utils and no scope of installing additional files . du-sh is the key , watch is to just do every 5 seconds. Pros : Works on any unix system Cons : No Progress Bar

To add another option, you can use cpv . It uses pv to imitate the usage of cp .

It works like pv but you can use it to recursively copy directories

There’s a ubuntu version in apt

pv is not an option because it won’t save permissions or any other file attributes. pv source > dest will give you a nice progress bar but the file dest is not going to be what you want.

find . -type f | pv -s $(find . -type f | wc -c) | xargs -i cp <> --parents /DEST/$(dirname <>) 

It finds all the files in the current directory, pipes that through PV while giving PV an estimated size so the progress meter works and then piping that to a CP command with the —parents flag so the DEST path matches the SRC path.

One problem I have yet to overcome is that if you issue this command

find /home/user/test -type f | pv -s $(find . -type f | wc -c) | xargs -i cp <> --parents /www/test/$(dirname <>) 

the destination path becomes /www/test/home/user/test/. FILES. and I am unsure how to tell the command to get rid of the ‘/home/user/test’ part. That why I have to run it from inside the SRC directory.

Источник

Advanced Copy Command with Progress Bar While Copying/Moving Files in Linux

As a Linux user, you may have found yourself in situations where you need to copy or move large files or directories from one location to another. This process can take some time, and it can be challenging to monitor progress of copy or move operation.

Fortunately, Linux provides several built-in commands that can help you copy or move files efficiently and also monitor progress of operation. In this article, we will discuss advanced copy command with progress bar while copying/moving files in Linux.

What is Advanced Copy Command?

The Advanced Copy (or «cp») command is a built-in Linux command used to copy files or directories from one location to another. This command comes pre-installed on most Linux systems, and it is widely used by system administrators, developers, and Linux enthusiasts.

The cp command provides several options that allow you to customize copy operation, such as preserving file attributes, recursive copying, and more. However, by default, cp command does not provide any progress bar while copying or moving files.

Why do we need a Progress Bar?

When you are copying or moving large files or directories, it can take a considerable amount of time, and it may be challenging to determine how much time is left for operation to complete. This is where a progress bar can be useful.

A progress bar is a graphical representation of copy or move operation’s progress. It shows percentage of operation that has been completed and how much time is left for operation to finish. This can be extremely helpful in situations where you need to monitor progress of copy or move operation.

Using «pv» command for Progress Bar

The «pv» command is a Linux utility that can be used to monitor progress of data through a pipeline. This command can be used in conjunction with other Linux commands, such as «cp,» to provide a progress bar while copying or moving files.

To use «pv» command with «cp» command, you will need to pipe output of «cp» command to «pv» command. Here’s an example −

$ cp -r /path/to/source /path/to/destination | pv -lep -s $(du -sb /path/to/source | awk '') >/dev/null

• «cp -r /path/to/source /path/to/destination» is copy command that copies source directory to destination directory.
• The pipe «|» sends output of «cp» command to «pv» command.
• «pv» is command that displays progress bar.
• «-lep» are options for «pv» command. «-l» enables rate limit, «-e» shows ETA (estimated time of arrival), and «-p» shows progress bar.
• «-s $(du -sb /path/to/source | awk »)» is option that specifies size of source directory in bytes. This option is required to calculate progress of copy operation accurately.
• «/dev/null» redirects output of «pv» command to null device, which discards it.

Using «rsync» Command for Progress Bar

Another command that can be used to copy or move files while displaying a progress bar is «rsync» command. «rsync» command is a powerful utility that can be used to synchronize files between two locations.

To use «rsync» command with a progress bar, you can use «-P» option. Here’s an example −

$ rsync -aP /path/to/source /path/to/destination

• «rsync» is command that synchronizes files between two locations.
• «-a» is option that enables archive mode, which preserves file attributes, permissions, timestamps, etc.
• «-P» is option that shows progress bar and displays transfer progress, transfer rate, ETA, and more.

Using «bar» Command for Progress Bar

Another utility that can be used to display a progress bar while copying or moving files is «bar» command. «bar» command is a simple utility that displays a progress bar in terminal.

To use «bar» command with «cp» command, you can pipe output of «cp» command to «bar» command. Here’s an example −

$ cp -r /path/to/source /path/to/destination | bar

• «cp -r /path/to/source /path/to/destination» is copy command that copies source directory to destination directory.
• The pipe «|» sends output of «cp» command to «bar» command.
• «bar» is command that displays progress bar.

In addition to utilities discussed above, there are several other options for displaying progress bars while copying or moving files in Linux. Some of these options include −

• «gcp» command − «gcp» command is a wrapper for «cp» command that adds a progress bar to copy operation. This command is not included in most Linux distributions, so you may need to install it manually.
• «progress» command − «progress» command is a Linux utility that can be used to display a progress bar for a variety of operations, including copying and moving files. This command can be used in conjunction with other commands, such as «cp» or «mv,» to display a progress bar.
• «bar-1.11» package − «bar-1.11» package is a set of shell scripts that can be used to display progress bars for various operations, including copying and moving files. This package can be installed on most Linux distributions.
• «dialog» command − «dialog» command is a Linux utility that can be used to create dialog boxes and menus in terminal. This command can be used to display a progress bar while copying or moving files.

When choosing a method for displaying progress bars, it’s essential to consider factors such as ease of use, compatibility with your Linux distribution, and reliability. While some methods may be more complicated than others, they may also provide more advanced features and functionality.

Overall, displaying a progress bar while copying or moving files in Linux can be an excellent way to monitor progress of operation and estimate time remaining for completion. By using commands and utilities discussed in this article, you can perform efficient file operations and manage your files effectively.

Conclusion

In conclusion, Linux provides several built-in commands and utilities that can be used to copy or move files and also display a progress bar to monitor progress of operation. «pv» command, «rsync» command, and «bar» command are some of utilities that can be used for this purpose.

While copying or moving files, it’s always a good practice to monitor progress of operation to ensure that it completes successfully without any errors. progress bar can be an excellent tool to monitor progress and estimate time remaining for operation to complete.

As a Linux user, it’s essential to be familiar with these commands and utilities to perform efficient file operations and manage your files effectively.

Источник