Linux copy files with progress

How can I move files and view the progress (e.g. with a progress bar)?

When moving large directories using mv , is there a way to view the progress (%)? The cp command on gentoo had a -g switch that showed the progress.

Measure pipe throughput in the shell might help, although I don’t know if it’s possible to easily use those tools when moving a file

i prefer rsync -aAX —info=progress2 (copy with progress info single totals bar, -a —archive mode: recursive, copies symlinks, keeps permissions, times, owner, group, device. -A acls -X extended attributes.) alias cpr=’rsync -aAX —info=progress2 ‘

14 Answers 14

There’s a new tool called progress that can find any descriptor related to a running command and show progress and speed: available here

outputs the stats for all running cp,mv etc. operations

WOW, I did not expect this to be so awesome! I simply typed progress after installing and it presented me with progress for the mv command I was currently waiting for. Thanks!

Thanks! When moving stuff as a root it seems you have to use sudo (same user I guess) — you can also use watch to say what is happening every 2 seconds: watch progress -w

Awesome program, but when moving e.g. a folder with many files it only shows the progress of the current file, not the whole folder in total. Instead of watch progress -w you can also simply use progress -m

IMHO, this is the most unix-way to solve the problem. Instead of patching coreutils or using tools which are not designated for specific purposes (like rsync for mv) or writing overkilling scripts, just use that tool written for the subject purpose with love.

I don’t like the idea to overwrite binaries from coreutil when there are simpler solutions, so here are mine:

rsync: Rsync copies files and has a -P switch for a progress bar. So if you have rsync installed, you could use a simple alias in your shells dotfile:

The downside is, that rsync is a little bit slower than cp, but you should measure this with time and decide for your self, I can live with it 🙂

Shell Script: A shell script can also create the progress bar. I found this a while ago on the net and I don’t remember the source:

#!/bin/sh cp_p() < strace -q -ewrite cp -- "$" "$" 2>&1 \ | awk '< count += $NF if (count % 10 == 0) < percent = count / total_size * 100 printf "%3d%% [", percent for (i=0;i<=percent;i++) printf "=" printf ">" for (i=percent;i <100;i++) printf " " printf "]\r" >> END < print "" >' total_size=$(stat -c '%s' "$") count=0 > 

% cp_p /home/echox/foo.dat /home/echox/bar.dat 66% [===============================> ] 

‘bar’ — ‘cat’ with ASCII progress bar

bar is a small shell script to display a process bar for all kind of operations (cp, tar, etc.). You can find examples on the project homepage.

Its also written for the bourne shell, so it will run nearby everywhere.

Note that rsync doesn’t replace mv (the OP is asking about) completely, especially when it comes to fast copies on copy-on-write filesystems like btrfs .

I have «nmv» aliased locally to rsync -a —stats —progress —remove-source-files . There are different flags you may want to set to control local caching, intermediate compression, etc., depending on your setup.

You can build a patched cp and mv which then both support the -g switch to show progress. There are instructions and patches at this page. However: The page instructs you to do

$ sudo cp src/cp /usr/bin/cp $ sudo cp src/mv /usr/bin/mv 

which overwrites the original cp and mv. This has two disadvantages: Firstly, if an updated coreutils package arrives at your system, they are overwritten. Secondly, if the patched version has a problem, they might break scripts relying on standard cp and mv. I would rather do something like this:

$ sudo cp src/cp /usr/local/bin/cpg $ sudo cp src/mv /usr/local/bin/mvg 

which copies the files to /usr/local/bin which is intended for user compiled programs and gives them a different name. So when you want a progress bar, you say mvg -g bigfile /mnt/backup and use mv normally.

Also you can do alias mvg=»/usr/local/mvg -g» then you only need to say mvg bigfile /mnt/backup and directly get the progress bar.

Unfortunately the patch on “this page” is no longer present and doesn’t appear to have been indexed in the Web Archive. There appear to be many patches with a similar name and function, could you edit your answer to clarify which one you meant?

I think github.com/atdt/advcpmv is a valid successor and it has the same patch so I changed the link to point to it.

@fschmitt: this link is no longer valid (404) but there is github.com/jarun/advcpmv . is this similar to what you linked to?

You can use pipe viewer command pv to show progress bar:

I often use this to copy a big file over a mounted network filesystem (combine with gzip and tar ). The drawback is that you can only copy one file and not directory. And you must give the new file a name, you can not just give destination directory like cp does. However copying is not pv ‘s purpose. It is a powerful tool and do much more than just copy file. See the homepage for more examples of pv .

A much better option is to use rsync -aP . If you want to mv instead, append the flag —remove-source-files . Add this to your .bashrc if you want to use the commands frequently:

alias rscp='rsync -aP' alias rsmv='rsync -aP --remove-source-files' 

The downside here is rsync only shows progress, not a progress bar.

tar cf — source-dir | pv | tar xf — -C out-dir will give you throughput of a whole directory. If you know the size and pass -s to pv it can give you progress as well.

If your goal is to move/copy a directory with progress bar, but avoiding non-terminal GUI, mc (Midnight Commander) is a good choice.

My solution is to use rsync . It can copy directories, remove the source files after a successful copy (thus «moving» them) and display progress reports, among many other features (most notably syncing partially copied directories and working over the network.)

Here is how I would move a directory with rsync, with progress report:

rsync -aP --remove-source-files $src $dst && rm -r $src 

Make sure $src does not end in a backslash, otherwise you will get a different meaning than that of GNU cp .

The rm -r at the end is needed because —remove-source-files only removes the source files, after each one is successfully copied over, not the directories. The && makes sure to run it only if rsync completes successfully. If you are paranoid, you can replace it with some kind of recursive rmdir , such as find $src -d -exec rmdir <> +

The only glitch is that the -P option shows progress for each file, not for the entire copy operation. This might be useless if you are trying to move a directory with many tiny files. The latest development version of rsync has an alternate flag —info=progress2 that shows progress for the entire operation.

Источник

Advanced Copy Command – Shows Progress Bar While Copying/Moving Files in Linux

Advanced-Copy is a powerful command line program that is very much similar, but a little modified version of the original cp command and mv tools.

This modified version of the cp command adds a progress bar along with the total time taken to complete while copying large files from one location to another.

This additional feature is very useful especially while copying large files, and this gives an idea to the user about the status of copy process and how long it takes to complete.

Install Advanced-Copy Command in Linux

The only way to install the Advanced-Copy utility in Linux systems is by building from sources using the following single curl command, which will download, patch, compile coreutils and generate the files: ./advcpmv/advcp and ./advcpmv/advmv .

# curl https://raw.githubusercontent.com/jarun/advcpmv/master/install.sh --create-dirs -o ./advcpmv/install.sh && (cd advcpmv && sh install.sh)

You might get the following error, during the installation process.

checking whether mknod can create fifo without root privileges. configure: error: in `/root/advcpmv/coreutils-9.1': configure: error: you should not run configure as root (set FORCE_UNSAFE_CONFIGURE=1 in environment to bypass this check) See `config.log' for more details

Run the following command on the terminal to fix that error and run the curl command again.

# export FORCE_UNSAFE_CONFIGURE=1 # curl https://raw.githubusercontent.com/jarun/advcpmv/master/install.sh --create-dirs -o ./advcpmv/install.sh && (cd advcpmv && sh install.sh)

Once, the installation completes, two new commands are created under ./advcpmv/advcp and ./advcpmv/advmv. You need to replace your original cp and mv commands with these two new commands to get the progress bar while copying files.

# mv ./advcpmv/advcp /usr/local/bin/cp # mv ./advcpmv/advmv /usr/local/bin/mv

Note: If you don’t want to copy these commands under standard system paths, you can still run them from the source directory like “./advcpmv/advcp” and “./advcpmv/advmv or create new commands as shown”.

# mv ./advcpmv/advcp /usr/local/bin/cpg # mv ./advcpmv/advmv /usr/local/bin/mvg

Show Progress Bar While Copying Files and Directories

If you want the progress bar to appear all the time while copying files and directories, you need to add the following lines to your ~/.bashrc file.

# echo alias cp '/usr/local/bin/advcp -g' >> ~/.bashrc # echo alias mv '/usr/local/bin/advmv -g' >> ~/.bashrc

You need to log out and log in again to get this to work correctly.

How to Use Advanced-Copy Command in Linux

The command is the same, the only change is adding the “-g” or “–progress-bar” option with the cp command. The “-R” option is for copying directories recursively.

Copy Files with Progress Bar

Here are example screen-shots of a copy process using the advanced copy command.

# cp -gR ubuntu-20.04.3-desktop-amd64.iso /home/tecmint/ OR # cp -R --progress-bar ubuntu-20.04.3-desktop-amd64.iso /home/tecmint/

Move Files with Progress Bar

Here is an example of the ‘mv‘ command with a screenshot.

# mv --progress-bar Songs/ /data/ OR # mv -g Songs/ /data/

Please remember, original commands are not overwritten if you ever need to use them or you’re not happy with the new progress bar and want to revert back to the original cp and mv commands. You can call them via /usr/bin/cp or /usr/bin/mv.

I really impressed with this new progress bar feature, at least I would know some information about copy operation time and exactly what’s going on.

You might also like:

Overall I can say, it is a really good tool to have in your pocket, especially when you are spending lots of time copying and moving files through the command line.

I am an experienced GNU/Linux expert and a full-stack software developer with over a decade in the field of Linux and Open Source technologies

Each tutorial at TecMint is created by a team of experienced Linux system administrators so that it meets our high-quality standards.

Related Posts

26 thoughts on “Advanced Copy Command – Shows Progress Bar While Copying/Moving Files in Linux”

“There are two methods to install Advanced-Copy utility in Linux systems, either you compile from sources or using pre-compiled binaries.” And for the second method. Nothing?! Zero! Zilch! Nada! Reply

alias cp='cp -gR' alias mv='mv -g'

alias cp='cpg -gR' alias mv='mvg -g'

@Janis, But the -g options worked for me on CentOS 6 version. On which Linux distribution version you’ve tried? let me give a try and see.. Reply

While running “make” i got this error, not sure what to do. CC lib/freadseek.o
lib/freadseek.c: In function ‘freadptrinc’:
lib/freadseek.c:67:3: error: #error “Please port gnulib freadseek.c to your platform! Look at the definition of getc, getc_unlocked on your system, then report this to bug-gnulib.”
#error “Please port gnulib freadseek.c to your platform! Look at the definition of getc, getc_unlocked on your system, then report this to bug-gnulib.”
^~~~~
make[2]: *** [Makefile:6662: lib/freadseek.o] Error 1 Reply

Thanks for updating but now it will not let me configure it I ran the command #export FORCE_UNSAFE_CONFIGURE=1 but still no luck can you help me here is what came up on terminal. roxanne-Satellite-C55-B coreutils-8.21 # ./configure
checking for a BSD-compatible install… /usr/bin/install -c
checking whether build environment is sane… yes
checking for a thread-safe mkdir -p… /bin/mkdir -p
checking for gawk… gawk
checking whether make sets $(MAKE)… yes
checking whether make supports nested variables… yes
checking for style of include used by make… GNU
checking for gcc… gcc
checking whether the C compiler works… no
configure: error: in `/home/roxanne/coreutils-8.21′:
configure: error: C compiler cannot create executables
See `config.log’ for more details
roxanne-Satellite-C55-B coreutils-8.21 # make
There seems to be no Makefile in this directory.
You must run ./configure before running ‘make’.
GNUmakefile:106: recipe for target ‘abort-due-to-no-makefile’ failed
make: *** [abort-due-to-no-makefile] Error 1 Reply

Источник