Linux file transfer network

Simple file transfer

@Steve_ Good point. Also -C will compress the files as they are being transferred, which can help over slow links.

I usually mount a directory through ssh via FUSE and sshfs.

$ sshfs name@server:/path/to/dir /path/to/mount/point 

$ fusermount -u /path/to/mount/point 

I use netcat (if I don’t need security)

nc -l -p 1234 < send_file # 'server' nc x.y.z.t 1234 >receive_file # 'client' 

The Network File System (NFS) allows a client node to perform transparent file access over the network. By using NFS, a client node operates on files residing on a variety of servers and server architectures, and across a variety of operating systems. File access calls on the client (such as read requests) are converted to NFS protocol requests and sent to the server system over the network.

You might require help from your Unix Admin to setup it first time but its very useful.

For quick-n-dirty *nix-to-*nix transfers, nothing comes close to NFS. A good admin can crank up both boxes and do transfers in less than 5 minutes, it’s near-transparent to the rest of the system, and it’s well-known and stable. +1 for something that doesn’t require banging out a command line every time to transfer something.

For one off file transfers, I usually use SFTP or an existing samba share.

For keeping in sync, I suggest you try rsync or unison (for 2-way synchronization)

Edit: scp would be better then sftp , since it would work on all SSH enabled hosts

For doing backups I often use rsync. If I want to backup onto a remote machine I’ll put a line in /etc/fstab to keep the remote machine mounted by NFS or CFIS (Samba).

192.168.0.101:/ /mnt/backup nfs rsize=8192,wsize=8192,timeo=14,intr 0 0 

Then have a line in my crontab using rsync.

rsync -av /home/user/sourcedir/ /mnt/backup/destinationdir > /home/user/backup.log 

Источник

How to Transfer Files Between Servers in Linux using SCP and FTP

Zaira Hira

Transferring files between machines is a very common operational task that you’ll do all the time as a developer.

Linux provides a number of utilities to transfer files. In this tutorial we will discuss FTP and SCP . Many automated scripts also deploy FTP or SCP to move files.

What is FTP?

FTP is a network protocol used for exchanging files over the network. It uses port 21. FTP enables you to access a remote system for exchanging files using the ftp command.

FTP Syntax

Here, host can either be the hostname or IP address of the remote host.

FTP Commands

FTP commands are similar to Linux commands. We’ll discuss some of these.

Command	Usage
open	Opens a remote connection with another computer.
get	Copies a file from the remote system to the local system.
put	Copies a file from the local system to a directory on the remote system.
mget	Transfers multiple files from the remote system to the local system’s current directory.
mput	Transfers multiple files from the local system to a directory on the remote system.
bye/quit	Prepares to exit the FTP environment.
close	Terminates the FTP connection.
ascii	Enables file transfer mode to ASCII
binary	Enables file transfer mode to binary.

How to Transfer Files via FTP

FTP offers two transfer modes: ASCII and Binary.

1. ASCII stands for American Standard Code for Information Interchange. It is used to transfer plain files such as text files.
2. Binary mode: Binary mode is used to transfer non-text files such as images.

The default transfer mode is ASCII.

Step 1 – Connect to FTP

In the example below, hostA is the remote host. You will be prompted for a username and password.

$ ftp hostA Connected to hostA. 220 hostA FTP server ready. Name (hostA:user): user 331 Password required for user. Password: password 230 User user logged in. Remote system type is LINUX. 

Once the connection is successful, you’ll notice the ftp> symbol in the beginning. Now we can run the FTP commands.

Step 2 – Choose file transfer mode

You can choose the mode (binary or ASCII) depending on your file type.

ftp> ascii 200 Type set to A.

Step 3 – Transfer files

We use the get command to transfer the file sample.txt from remote FTP server to local machine.

ftp> get sample.txt 200 PORT command successful. 150 Opening ASCII mode data connection for sample.txt (22 bytes). 226 Transfer complete. local: sample.txt remote: sample.txt 22 bytes received in 0.012 seconds (1.54 Kbytes/s)

Step 4. End the session

ftp> bye 221-You have transferred 22 bytes in 1 files. 221-Total traffic for this session was 126 bytes in 2 transfers. 221-Thank you for using the FTP service on hostA. 221 Goodbye.

How to Transfer Multiple Files via FTP

To transfer files in bulk, there are two commands: mget and mput .

You use mget to download the files, whereas you use mput to upload the files.

ftp> mget sample_file.1 sample_file.2

ftp> mput sample_file.1 sample_file.2

All the steps we just learned can be placed in an executable file and be scheduled. You can find the code for automation here.

What is SCP?

SCP stands for Secure Copy. It uses SSH and port 22. The data transferred through SCP is encrypted and sniffers can’t access it. This makes SCP very secure.

• Transfer files from local machine to remote host.
• Transfer files from remote host to local machine.

SCP syntax

Let’s explore the syntax of SCP.

scp [FLAG] [user@]SOURCE_HOST:]/path/to/file1 [user@]DESTINATION_HOST:]/path/to/file2 

• [FLAG] specifies the options that can be given to SCP. Here are some details about flags:

• [user@]SOURCE_HOST is the source machine.
• [user@]DESTINATION_HOST:] is the destination machine.

Note: To transfer files via SCP, credentials must be known and the user should have the permissions to write.

How to Transfer Files from Local Machine to Remote Host via SCP

To transfer files to a remote host, use the command below:

scp source_file.txt remote_username@10.13.13.11:/path/to/remote/directory

In the command above, source_file.txt is the file to be copied. Remote_username is the username for remote host 10.13.13.11 . After : the destination path is specified.

remote_username@10.13.13.11's password: source_file.txt 100% 0 0.0KB/s 00:00 

The file source_file.txt will now be placed in /path/to/remote/directory .

To copy directories, use the -r flag as demonstrated below.

scp -r /local/directory remote_username@10.13.13.11:/path/to/remote/directory

How to Transfer Files from Remote Host to Local Machine via SCP

To transfer files from a remote host to a local machine, use the command below:

scp remote_username@10.13.13.11:/remote/source_file.txt /path/to/local/directory

Be extra careful when transferring files as SCP overwrites the already existing files.

Wrapping up

In this tutorial, you learned how to transfer files and directories using FTP and SCP via command line.

When automated, these commands serve even greater purposes in data warehousing, ETL (Extract, Transform, Load), reporting, archiving and bulk file processing. Do give these commands a try. Let’s connect on Twitter.

Источник

How to Transfer Files Between Two Computers using nc and pv Commands

Hi fellow Linux readers, I’m bringing you yet another great article from our Linux Tips and Tricks series, this time we will be using two lesser known Linux utilities that you should must know about.

This article will explain how do you transfer files between two Linux computers using nc (networking utility) and pv (pipe viewer) commands, before moving further let me explain what are these two commands.

nc stands for Netcat and often point out as “Swiss Army knife” is a networking tool used for network debugging and investigation and also it is used for creating network connections using TCP or UDP, port scanning, file transfer and more. It is created to be a dependable back-end and specially used in programs and scripts, since it can generate almost any kind of network connection and has a number of built-in features.

pv in short Pipe Viewer is a terminal based tool for monitoring progress of data send through a pipeline, it allows a user to see the progress of data with progress bar, shows time elapsed, percentage completed, current throughput rate, total data transferred, and Estimated Time to complete the process.

Let’s now move further and see how we can combine both commands to transfer files between two Linux computers, for the purpose of this article we will be using two Linux machines as follows:

Machine A with IP : 192.168.0.4 Machine B with IP : 192.168.0.7

Note: I strongly advise not to use netcat to send and receive data over pubic network, as it doesn’t use any logins or authentication, the only requirement is the client IP and listening port number and the data send over network is not secured, thus, it always reserved for advanced Linux users and preferred to use on protected local network.

Situations where security of data is more important, then always use rsync over SSH or scp over SSH.

Now let’s start with some real easy example of nc and pv commands, but before doing that both utilities must installed on the system, if not install them using your respective distribution package manager tool as suggested:

# yum install netcat pv [On RedHat based systems] # dnf install netcat pv [On Fedora 22+ versions] # apt-get install netcat pv [On Debian and its derivatives]

How to Transfer Files Between Two Linux Machines?

Let’s assume that you want to send one large file called CentOS-7-x86_64-DVD-1503.iso from computer A to B over network, the quickest way to achieve this using nc a network utility used to send files over TCP network, pv to monitor the progress of data and tar utility to compress data to improve transfer speed.

On Linux Machine A

First login into the machine ‘A‘ with IP address 192.168.0.4 and run the following command.

# tar -zcf - CentOS-7-x86_64-DVD-1503.iso | pv | nc -l -p 5555 -q 5

Let me explain the options used in above command:

1. tar -zcf = tar is a tape archive utility used to compress/uncompress archive files and arguments -c creates a new .tar archive file, -f specify type of the archive file and -z filter archive through gzip.
2. CentOS-7-x86_64-DVD-1503.iso = Specify the file name to send over network, it can be file or path to a directory.
3. pv = Pipe Viewer to monitor progress of data.
4. nc -l -p 5555 -q 5 = Networking tool used for send and receive data over tcp and arguments -l used to listen for an incoming connection, -p 555 specifies the source port to use and -q 5 waits the number of seconds and then quit.

On Linux Machine B

Now login into machine ‘B‘ with IP address 192.168.0.7 and run the following command.

# nc 192.168.1.4 5555 | pv | tar -zxf -

That’s it, the file gets transferred to computer B, and you’ll be able to watch how quick the operation was performing. There are tons of more other great usage of nc (not covered yet, but will write about it soon) and pv (we already covered a detail article on this here) commands, if you know any example, please let us know via comments!

Источник