Linux backup with cron

Automatic Website Backups with Tar, Cron, and Rsync on Ubuntu 20.04

This guide describes how to create automated backups of a web server’s document root and Apache configuration files using the utilities tar , rsync , and cron . The example schedule runs daily, with backup retention for one year.

Prerequisites

• A fully-updated Ubuntu Linux 20.04 server, running Apache.
  • (Referred to as the «web server«)
  • (Referred to as the «offsite server«)

  You can adapt this guide for other web servers and Linux distributions with slight modifications. The commands used are standard for most Linux distros.

  The 3-2-1 Rule

  The 3-2-1 backup rule states that you need three copies of your data. In addition to the primary working copy, you need two backups, stored on different types of storage media, with one of the copies offsite. With this system, you’re highly unlikely to lose your data.

  1. Backup Preparation

  Before creating a backup, ensure you have sufficient disk space on the web server.

  Compare this to the size of your website root and Apache configuration files and any other directories you wish to back up:

  $ du -sh /var/www $ du -sh /etc/apache2 

  Once you have confirmed enough space is available to perform the backup, create a directory to contain the local backup.

  2. Perform a Backup

  Create a backup of the web server’s document root and Apache configuration directory with tar . Name this file web-server.tar.gz .

  $ sudo tar -cvzpf /var/web_backup/web-server.tar.gz -C /var/ www -C /etc/ apache2 

  The arguments ( -cvpzf -C ) in the command specify the following:

  • c — Create archive
  • v — Verbose
  • z — Compress the file using gzip
  • p — Preserve permissions of the files
  • f — Path and filename of the archive
  • C — Change directory

  Note the space after /var/ and /etc/ in the command before specifying the directory’s name to back up. Include any other directories you wish to back up with additional -C arguments in the command.

  How to exclude files

  To exclude a file or directory from your archive, use —exclude=path/to/file/ . For example, if you want to exclude a directory /var/www/example.com/junk/ , use:

  $ sudo tar -cvpzf /var/web_backup/web-server.tar.gz --exclude=www/example.com/junk -C /var/ www -C /etc/ apache2 

  Note: Don’t include the full path when specifying what to exclude, use the relative path. For instance, use www/example.com/junk rather than /var/www/example.com/junk . Use multiple —exclude=path/to/file/ directives when excluding multiple files and directories. You may also use an exclude file if you have many directories to exclude.

  3. Test a Restore

  Before automating the process, make sure you can restore the files. First, create a test directory for the restore.

  $ sudo mkdir /var/web_backup/restored/ 

  Restore the archive to the directory.

  $ sudo tar -xvzpf /var/web_backup/web-server.tar.gz -C /var/web_backup/restored/ 

  The extract command has some new arguments:

  Verify the restore was successful by listing the contents.

  $ ls -lh /var/web_backup/restored/ 

  Once verified, remove the test directory:

  $ sudo rm -r /var/web_backup/restored/ 

  4. Automate with Cron

  The cron utility schedules commands to run at specific times. This example will schedule daily backups at 02:00 and keep one backup for each day of the week, and use the date command to name the archive with the day of the week it was created.

  $ sudo tar -cvpzf /var/web_backup/web-server.`date +\%a`.tar.gz -C /var/ www -C /etc/ apache2 

  The +\%a variable specifies the date should be returned as an abbreviated day of the week. For example, «Mon», Tue», etc. The tar command will overwrite the daily archive from the previous week.

  For a full list of format characters supported by the date command, refer to the man pages.

  Add a job to crontab

  The cron schedules are stored in a file named crontab. The typical crontab entry begins with five values (or asterisks), followed by a command. The values tell cron when to execute the command, and an asterisk means «all».

  * * * * * command to be executed - - - - - | | | | | | | | | ----- Day of week (0 - 7) (Sunday=0 or 7) | | | ------- Month (1 - 12) | | --------- Day of month (1 - 31) | ----------- Hour (0 - 23) ------------- Minute (0 - 59) 

  Add the tar command to the bottom of the file, including the schedule at the beginning:

  00 02 * * * sudo tar -cvpzf /var/web_backup/web-server.`date +\%a`.tar.gz -C /var/ www -C /etc/ apache2 

  This schedules cron to run the command at 02:00, every day of any month.

  12 month retention

  To keep monthly backups, use another schedule that appends the filename with an abbreviated month name using the +\%b parameter to the command. To test date commands before inserting them into the filename, use echo to print them:

  Add this to a new line in crontab :

  0 0 1 * * sudo tar -cvpzf /var/web_backup/web-server.`date +\%b`.tar.gz -C /var/ www -C /etc/ apache2 

  This schedules cron to run the monthly backup at 00:00, on the first day of the month. After a year, older archives are overwritten.

  5. Back up Offsite

  To complete the 3-2-1 backup rule, the offsite server will use rsync , to download the backups from the web server. Ensure both the web server and the offsite server have rsync installed. It is installed by default on Ubuntu systems, and can be manually installed with:

  • a — Archive, sync recursively while preserving symbolic links, file permissions, etc
  • z — Compress file data during transfer
  • P — Short for —partial and —progress , keep partially transferred files if interrupted and display the progress of the transfer
  • —delete — Delete extraneous files from the destination directory

  Use caution with —delete . Test your command with the —dry-run option first to prevent data loss.

  Excluding files

  You may exclude files from rsync . To exclude a file from being transferred, use the —exclude=relative/path/to/file/ argument in your command. For example, if you want to exclude Wednesday’s backup from being transferred:

  $ rsync -azP --delete --exclude=web-server.Wed.tar.gz username@remote_host:/var/web_backup/ /path/to/offsite/server_backup 

  Repeat —exclude=relative/path/to/file/ for each file or directory you wish to exclude.

  Note: Don’t include the full path when specifying what to exclude, exclude paths need to be relative to the source path. For instance, to exclude Wednesday’s backup stored in /var/web_backup/daily/ , use daily/web-server.Wed.tar.gz , not /var/web_backup/daily/web-server.Wed.tar.gz .

  6. Passwordless Rsync Login

  To automate the process, configure rsync to connect to the web server from the offsite server without a password.

  On the offsite server, create a key pair for rsync :

  $ ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/your_home/.ssh/id_rsa): 

  $ ssh-copy-id -i ~/.ssh/id_rsa_rsync username@remote_host 

  $ rsync -azP --delete -e 'ssh -i ~/.ssh/id_rsa_rsync' username@remote_host:/var/web_backup/ /path/to/offsite/server_backup 

  $ rsync -azP --delete -e 'ssh -vi ~/.ssh/id_rsa_rsync' username@remote_host:/var/web_backup/ /path/to/offsite/server_backup 2>&1 | grep "Sending command" Output: debug1: Sending command: rsync --server --sender -vlogDtprz . /var/web_backup/ 

  $ sudo nano ~/.ssh/authorized_keys 

  ssh-rsa AAAAB3Nza. [many characters]. LiPk== user@example.com 

  command="rsync --server --sender -vlogDtprz . /var/web_backup/",no-pty,no-agent-forwarding,no-port-forwarding ssh-rsa AAAAB3Nza. [many characters]. LiPk== user@example.com 

  $ rsync -azP --delete -e 'ssh -i ~/.ssh/id_rsa_rsync' username@remote_host:/var/web_backup/ /path/to/offsite/server_backup 

  $ ssh -i ~/.ssh/id_rsa_rsync username@remote_host 

  7. Automate the Process

  Open crontab on the offsite server.

  Append the rsync command to the bottom of the file.

  00 03 * * * rsync -azP --delete -e 'ssh -i ~/.ssh/id_rsa_rsync' username@remote_host:/var/web_backup/ /path/to/offsite/server_backup 

  This schedules cron to run the rsync command at 03:00 each day.

  Conclusion

  You have a working 3-2-1 backup strategy! Your website’s automatic backups are made daily and sent to an offsite location with retention for one year.

  Want to contribute?

  You could earn up to $600 by adding new articles.

  Источник

  Schedule backup in Linux (rsync + cron)

  Creating automatic backup will not only save the time you would spend creating the backups manually, but it may save you days, weeks, months or even years of work when something goes wrong and you lose some important data. Everyone knows about this but still a few users schedule an automatic backup in Linux. Even I have postponed configuring scheduled backups in my new KDE Neon install for a few months already. Luckily, nothing wrong happened with my data during this time.

  I have just configured my system for automatic daily, weekly, and monthly backups with rsync and I would like to share how I did that. Keep in mind that rsync is a command line tool if you prefer graphical backup programs have a look at LuckyBackup or CloudBerry Backup.

  How to schedule rsync

  Rsync is installed on all popular Linux systems and you can easily configure it to run on schedule in the background. The most popular and simplest way to do that is to use the cron job scheduler. Cron is installed and configured on all Debian-based systems like Ubuntu, Linux Mint, KDE Neon, etc.

  You simply need to add the schedule of the rsync commands you want to run to the crontab file:

  00 12 * * * rsync -aAX --delete --exclude '*.Trash-1000' /source/ /backup/daily 00 15 * * 5 rsync -aAX --delete --exclude '*.Trash-1000' /source/ /backup/weekly 00 16 1 * * rsync -aAX --delete --exclude '*.Trash-1000' /source/ /backup/monthly_$(date +%Y%m%d) 

  

  I am not going to explain the rsync command. I have done it already in this post. Here, I will only explain the schedule format. You can see the schedule format from the comments in the crontab file. The columns abbreviation means the following:

  Hence, my schedule command means that rsync will run at 12:00 daily, at 15:00 on Friday weekly, and at 16:00 on the first day of every month. To configure the time of your automatic backup in Linux, change these time points as you need.

  From the above command, you can also see that daily and weekly backups will be overwritten, while all monthly backups will be kept with the date in their names. For example, my backup created on April 1, 2019, has the following name monthly_20190401 . If you have a limited amount of space, you can set monthly backups to be overwritten too. Just removed _$(date +%Y%m%d) .

  I think this is the most reasonable way to automatically backup Linux. You can also compress monthly and weekly backups, but in my experience, compression is not practical. First, it takes a lot of time to run. Second, looking for specific files in a compressed backup is slow.

  NOTE that your computer must be on during the scheduled time to execute the rsync commands. Cron will not run missed jobs. To run missed jobs, look at anacron.

  Where to backup

  You should never store your backup on the same hard drive which you backup from. If it breaks, you will lose all data. I recommend using a separate hard drive either installed in your computer or connected externally through the USB.

  I personally have a WD 4 TB hard drive installed on my system that is used for automatic backup, and I also copy monthly backups manually to an external WD 20 TB RAID hard drive. If my computer gets stolen, I will still have a backup of all data on my external hard drive.

  You can purchase a hard drive using my Amazon page, where I share the hard drives I use. They all are Linux compatible.

  CONCLUSION

  Automatic backup in Linux takes only three lines of simple code in your crontab file, but it can be a lifesaver one day. So, do not wait any more, go and configure your automatic backup in Linux.

  Average Linux UserFollow I am the founder of the Average Linux User project, which is a hobby I work on at night. During the day I am a scientist who uses computers to analyze genetic data.

  Источник

  Читайте также:  Notebook fan control linux