Linux copy with progress bar

Progress bar for scp command

Is there any way to display a progress bar while copying from server to local (or vice versa) using scp ?

4 Answers 4

I don’t think that this can be done with scp . Last time I needed something like this i.e. progress shown, I used rsync instead. It shows progress in a bar-like manner. See if it works for you.

You will need to use the —progress option of rsync . You can use the following command:

rsync -r -v --progress -e ssh user@remote-system:/address/to/remote/file /home/user/ 

I was going to post something similar but when I tested it, I just got 2741851 0% 700.39kB/s 0:17:21 and no graphical progress bar (what I think the OP wants).

@Oli: I think its because you are copying a very small file. Copying ends before rsync can show progress. If you copy a bigger file then you should get a progress bar.

For anybody who was looking for cp with progress bar, rsync works great locally, so this answers that question also! (Just leave off the -e ssh user@remote-system: for a local copy and man rsync explains the many, many options)

If you want to connect to a different SSH port than the default, you can use something like rsync -avz —progress -e ‘ssh -p 1223’ root@google.com:/foobar.txt ./my-local-copy.txt

@fuero man scp : » -3 ​ ​ ​ ​ Copies between two remote hosts are transferred through the local host. Without this option the data is copied directly between the two remote hosts. Note that this option disables the progress meter.«

As of 2018, progress and ETA are shown by default and could be disabled by -q

With this, do you mean overall progress or on a per-file basis? I only see it on a per-file basis in my freshly installed Debian 9.

For some reason I can see the progress when I run scp in a terminal, but when it is running as part of a cloud-init script, I don’t see any progress (with curl I did see progress). any idea why? And how to show progress even in a cloud-init process?

@Drachenfels at least as of the latest macOS (Ventura at the moment), it does show the progress and ETA by default.

I don’t know how to do this in a command line. I’m sure it’s possible but there is a graphical method for doing this.

Nautilus (the default file browser in Ubuntu) can mount ssh/sftp servers. They act like a local filesystem after that and you can copy files around like you normally would. And you get the usual progress bar that you would with a normal copy.

Look under the File menu for Connect to server.

You must log in to answer this question.

Linked

Related

Hot Network Questions

Subscribe to RSS

To subscribe to this RSS feed, copy and paste this URL into your RSS reader.

Site design / logo © 2023 Stack Exchange Inc; user contributions licensed under CC BY-SA . rev 2023.7.14.43533

Ubuntu and the circle of friends logo are trade marks of Canonical Limited and are used under licence.

By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy.

Источник

Advanced Copy Command – Shows Progress Bar While Copying/Moving Files in Linux

Advanced-Copy is a powerful command line program that is very much similar, but a little modified version of the original cp command and mv tools.

This modified version of the cp command adds a progress bar along with the total time taken to complete while copying large files from one location to another.

This additional feature is very useful especially while copying large files, and this gives an idea to the user about the status of copy process and how long it takes to complete.

Install Advanced-Copy Command in Linux

The only way to install the Advanced-Copy utility in Linux systems is by building from sources using the following single curl command, which will download, patch, compile coreutils and generate the files: ./advcpmv/advcp and ./advcpmv/advmv .

# curl https://raw.githubusercontent.com/jarun/advcpmv/master/install.sh --create-dirs -o ./advcpmv/install.sh && (cd advcpmv && sh install.sh)

You might get the following error, during the installation process.

checking whether mknod can create fifo without root privileges. configure: error: in `/root/advcpmv/coreutils-9.1': configure: error: you should not run configure as root (set FORCE_UNSAFE_CONFIGURE=1 in environment to bypass this check) See `config.log' for more details

Run the following command on the terminal to fix that error and run the curl command again.

# export FORCE_UNSAFE_CONFIGURE=1 # curl https://raw.githubusercontent.com/jarun/advcpmv/master/install.sh --create-dirs -o ./advcpmv/install.sh && (cd advcpmv && sh install.sh)

Once, the installation completes, two new commands are created under ./advcpmv/advcp and ./advcpmv/advmv. You need to replace your original cp and mv commands with these two new commands to get the progress bar while copying files.

# mv ./advcpmv/advcp /usr/local/bin/cp # mv ./advcpmv/advmv /usr/local/bin/mv

Note: If you don’t want to copy these commands under standard system paths, you can still run them from the source directory like “./advcpmv/advcp” and “./advcpmv/advmv or create new commands as shown”.

# mv ./advcpmv/advcp /usr/local/bin/cpg # mv ./advcpmv/advmv /usr/local/bin/mvg

Show Progress Bar While Copying Files and Directories

If you want the progress bar to appear all the time while copying files and directories, you need to add the following lines to your ~/.bashrc file.

# echo alias cp '/usr/local/bin/advcp -g' >> ~/.bashrc # echo alias mv '/usr/local/bin/advmv -g' >> ~/.bashrc

You need to log out and log in again to get this to work correctly.

How to Use Advanced-Copy Command in Linux

The command is the same, the only change is adding the “-g” or “–progress-bar” option with the cp command. The “-R” option is for copying directories recursively.

Copy Files with Progress Bar

Here are example screen-shots of a copy process using the advanced copy command.

# cp -gR ubuntu-20.04.3-desktop-amd64.iso /home/tecmint/ OR # cp -R --progress-bar ubuntu-20.04.3-desktop-amd64.iso /home/tecmint/

Move Files with Progress Bar

Here is an example of the ‘mv‘ command with a screenshot.

# mv --progress-bar Songs/ /data/ OR # mv -g Songs/ /data/

Please remember, original commands are not overwritten if you ever need to use them or you’re not happy with the new progress bar and want to revert back to the original cp and mv commands. You can call them via /usr/bin/cp or /usr/bin/mv.

I really impressed with this new progress bar feature, at least I would know some information about copy operation time and exactly what’s going on.

You might also like:

Overall I can say, it is a really good tool to have in your pocket, especially when you are spending lots of time copying and moving files through the command line.

I am an experienced GNU/Linux expert and a full-stack software developer with over a decade in the field of Linux and Open Source technologies

Each tutorial at TecMint is created by a team of experienced Linux system administrators so that it meets our high-quality standards.

Related Posts

26 thoughts on “Advanced Copy Command – Shows Progress Bar While Copying/Moving Files in Linux”

“There are two methods to install Advanced-Copy utility in Linux systems, either you compile from sources or using pre-compiled binaries.” And for the second method. Nothing?! Zero! Zilch! Nada! Reply

alias cp='cp -gR' alias mv='mv -g'

alias cp='cpg -gR' alias mv='mvg -g'

@Janis, But the -g options worked for me on CentOS 6 version. On which Linux distribution version you’ve tried? let me give a try and see.. Reply

While running “make” i got this error, not sure what to do. CC lib/freadseek.o
lib/freadseek.c: In function ‘freadptrinc’:
lib/freadseek.c:67:3: error: #error “Please port gnulib freadseek.c to your platform! Look at the definition of getc, getc_unlocked on your system, then report this to bug-gnulib.”
#error “Please port gnulib freadseek.c to your platform! Look at the definition of getc, getc_unlocked on your system, then report this to bug-gnulib.”
^~~~~
make[2]: *** [Makefile:6662: lib/freadseek.o] Error 1 Reply

Thanks for updating but now it will not let me configure it I ran the command #export FORCE_UNSAFE_CONFIGURE=1 but still no luck can you help me here is what came up on terminal. roxanne-Satellite-C55-B coreutils-8.21 # ./configure
checking for a BSD-compatible install… /usr/bin/install -c
checking whether build environment is sane… yes
checking for a thread-safe mkdir -p… /bin/mkdir -p
checking for gawk… gawk
checking whether make sets $(MAKE)… yes
checking whether make supports nested variables… yes
checking for style of include used by make… GNU
checking for gcc… gcc
checking whether the C compiler works… no
configure: error: in `/home/roxanne/coreutils-8.21′:
configure: error: C compiler cannot create executables
See `config.log’ for more details
roxanne-Satellite-C55-B coreutils-8.21 # make
There seems to be no Makefile in this directory.
You must run ./configure before running ‘make’.
GNUmakefile:106: recipe for target ‘abort-due-to-no-makefile’ failed
make: *** [abort-due-to-no-makefile] Error 1 Reply

Источник

how to create progress bar during copy files

In short, you won’t find cp native functionality for progress bar output. Why? Many reasons. However, you have some options:

Use a different tool. rsync , as mentioned by @user1404316 has —progress :

rsync -P largeFile copyLocation 

# In one shell $ cp largeFile copyLocation # In another shell $ progress -m [ 4714] cp /home/hunteke/largeFile 1.1% (114 MiB / 10.2 GiB) # -m tells progress to continually update 

gcp largeFile copyLocation 

curl -o copyLocation file:///path/to/largeFile 

@yael mild semantic difference. Yours will continue where a previous copy left off. The mentioned version mimics cp ‘s standard overwrite behavior.

I download the pv , and I see that copy is very fast it take much less then cp , so my question is dose pv copy the source to target with high reliability

Not knowing your exact command, I’m surprised at the speed difference. The underlying mechanism doing the copying in both cases is the kernel. cp may be doing some extra CRC or other checks, but I doubt it. What cp will do that pv won’t, especially with -a , is maintain stat properties, like time modified, ownership (if you’re running as root ), etc.

rsync is a copy program that has great features for copying huge files, including a —progress feature, and it has a form of in-transit compression to reduce the data in transit (and thus saving time for the copy):

rsync --progress file1 file2 

rsync also has features to resume a copy that was interrupted, maybe due to a network disconnection. You would need to start it with options —partial —append-verify . The only downside that I can think of to using rsync for copying over a network is that it needs to have been installed on both computers, ie. the sender and the receiver. See its man page for details and all its other features.

If you can’t install rsync for some reason, or don’t want to, a good simple alternative is pv , a pipe viewer command, that does exactly what you are asking for.

pv also includes many visualization options. See its man page for all the goodies available. Here are some of the commonly used options:

-p, --progress -t, --timer -e, --eta -r, --rate -a, --average-rate -b, --bytes 

Источник