Linux: Tape backup for a home network
In another article, I have discussed why tapes are still the most efficient form of backup. In this article, I detail how I used tapes in a Linux based home network for backup purposes, using a Seagate/Certance STT20000A IDE ATAPI Internal Travan TR-5 Tape drive. There are also several resources listed.
Background and Historical Notes
I have been backing up my personal stuff on tape for about 15 years. I used to make copies of my home directory (on UNIX, at work) to a QIC tape that had a whopping 60 MB capacity, running on an NCR TOWER (Motorola 680×0 CPU). Later, I would use 150 MB QIC tapes on the same architecture.
I then migrated to the same drives but on NCR System 3000 (Intel x86 CPU), and had to do byte swapping conversion on the data:
dd if=/dev/rmt/c0s0 conv=swab | cpio -icvmuld
Since QIC drives were expensive, this was out of my reach at home. But there were plenty of those at work, and my stuff was mainly on my machine at work. This high cost factor alone contributed to my being delayed in running UNIX/Linux at home for years.
When I started using Linux at home, I bought an HP Colorado 2.5GB/5GB Travan IDE tape drive. It worked well and I backed up stuff on it when my hard drive was just 2GB. I used several tapes, and backed up every week or two, normally overnight.
The tape drive was recognized as an IDE Tape, so the device was /dev/ht0, so in the dmesg output, or in /var/log/messages you should see something like:
ide-tape: hdd ht0, 600KBps, 14*26kB buffer, 2600kB pipeline, 310ms tDSC, DMA
A Real Life Example
In 2000, my home was broken into and the PC stolen! Those tapes came in handy with all my data on them. I only lost about a month’s worth of data.
I bought a used Seagate 10GB/20GB Travan IDE tape drive off someone selling it on eBay for about 100$US, including several used tapes. I restored my home directory and everything was good to go. If I had bought another hard disk and copied stuff to it, all my data would have been gone forever! Remember that next time you say «buy another disk!»
All the computers on the home network share the server’s drive using either Samba (for Windows clients) or NFS (for Linux clients). Each user is responsible for making sure that their files end up on the server, otherwise, they are not backed up.
I use a set of 6 tapes for weekly backup and 3 for monthly backups, all using cpio. The monthly ones are kept offsite, at a friend/co-worker’s house. Even with removable hard disks, you can’t justify the cost of 9 hard disks!
The Linux cron facility is used to start the backup in the early hours of the weekend. I even have another cron job the day before to check that the tape is inserted in the drive and that it has been rewind to the beginning of tape. The entire process is unattended, except for an email telling me the backup is done, and to change the tape, or to remind me of the monthly backup.
My backup is about 7 GB now for the home server, which contains my kids homework, my files, digital pictures, CVS repository, ..etc. So there still room on the tapes.
Configuration
In order to configure it with Linux, you need to specify the kernel argument hdx=ide-scsi (where x is the letter of the IDE position for that drive). You can specifiy this argument in the lilo or grub configuration file.
You will know that the tape is correctly installed and that it is recognized by the Linux kernel, when you run the dmesg command or view the /var/log/messages log, and see the following:
First a message that the IDE driver has recognized the drive.
hdd: Seagate STT20000A, ATAPI TAPE drive
Then, you should see the SCSI over IDE layer loaded, and it should recognize the tape as well:
SCSI subsystem initialized
scsi0 : SCSI host adapter emulation for IDE ATAPI devices
Vendor: Seagate Model: STT20000A Rev: 8.25
Type: Sequential-Access ANSI SCSI revision: 02
Finally, you should see the SCSI Tape driver loaded:
st: Version 20040122, fixed bufsize 32768, s/g segs 256
Attached scsi tape st0 at scsi0, channel 0, id 0, lun 0
st0: try direct i/o: yes, max page reachable by HBA 163838
The tape device is now known as /dev/st0 to the tape programs.
You can also use /dev/nst0 if you do not want the tape to rewind on close. This is useful if you want to be creative and «append» backups on a tape.
Note: You may see the following messages in the output of dmesg or in /var/log/messages. Do not be alarmed. There is no ill effect from there, and they are harmless:
Jul 14 12:53:36 zzzz kernel: st0: Error with sense data: Current st0: sense key Illegal Request
Jul 14 12:53:36 zzzz kernel: Additional sense: Invalid command operation code
Backup and Restore Commands
You can then use the following commands:
To get the tape status, do the following:
mt -f /dev/st0 status
SCSI 2 tape drive:
File number=0, block number=0, partition=0.
Tape block size 512 bytes. Density code 0x47 (TR-5).
Soft error count since last status=0
General status bits on (41010000):
BOT ONLINE IM_REP_EN
Note the status bits at the end of the output. These are useful, but are often cryptic. Here is an explanation of the most important ones:
| Status Bit | Description |
|---|---|
| BOT | The tape is positioned at the beginning of the first file. |
| EOT | A tape operation has reached the physical End Of Tape. |
| EOF | The tape is positioned just after a filemark. |
| WR_PROT | The tape (or drive) is write-protected. For some drives this can also mean that the drive does not support writing on the current medium type. |
| ONLINE | The drive has a tape in place and ready for operation. |
| DR_OPEN | Door is open. Depending on the type of drive, this usually means that the drive does not have a tape in place. |
| IM_REP_EN | Immediate report mode. This bit is set if there are no guarantees that the data has been physically written to the tape when the write call returns. It is set to zero only when the driver does not buffer data and the drive is set not to buffer data. |
| SM | The tape is currently positioned at a setmark. DDS specific. |
| EOD | The tape is positioned at the end of recorded data. DDS specific. |
| D_6250 D_1600 D_800 | This «generic» status information reports the current density setting for 9-track 1/2 inch tape drives only. |
If you do not have a tape cartridge inserted in the tape drive, you will see:
If you have a tape cartridge in the tape drive, but the write protect tab is on, you will see:
BOT WR_PROT ONLINE IM_REP_EN
In order to rewind the tape, do the following:
# Rewind the tape
mt -f /dev/st0 rewind
For actual backup, I do not use dump since it only dumps one filesystem, and does not cross the boundary. Therefore I will need two tapes to backup my system, which is kind of a waste. I could actually make the backups back to bak, but this complicates things if I am looking to read info from the tape, specially in the panic of a recovery situation. It is best to backup everything using cpio, with relative pathnames, and then restore whatever you need to wherever you need, even if partitions are different.
Therefore, I prefer to use cpio. Even tar is a bit primitive compared to cpio, since it has many nice features, such as preserving the modification time.
To backup, use the following command:
# Backup a directory using cpio crc header format
cd /home
find . -print | cpio -ovH crc -O /dev/st0
The -H crc option specifies that the backup will be in a portable format that can be read on other UNIX machines, and not only Linux.
If you are doing a full system backup, then you have to exclude some directories, such as /proc, /dev and /mnt.
To list the tape contents, you can do:
To restore the tape contents, you can do:
Check the appropriate manual pages for the mt and cpio commands for more details (links below in the Resources section). You may also want to check the dd manual page as well, since you can do some neat tricks with it.
So, I will stay with tape for the forseable future, until optical technology catches up with disk size increases.
Resources and Links
- Thanks to LinuxQuestions, the manual pages for Linux commands are available online, in HTML format. Here are the pages for
- cpio
- tar
- dump
- mt
- dd
15 полезных Linux команд для управления ленточными библиотеками
Ленточные устройства на регулярной основе следует использовать для архивирования файлов или для передачи данных от одного сервера к другому. Как правило, ленточные устройства все подключены к Unix системе. Резервные копии следует выполнять на диски (например в облако) и ленточные устройства. В этой статье мы рассмотрим:
- Название ленточных устройств
- Команды для управления лентами
- Основные команды резервного копирования и восстановления с ленточных устройств
Зачем выполнять резервное копирование?
- Резервное копирование очень важно:
- Возможность восстановления при отказе работы дисковой подсистемы
- Случайное удаление файлов
- Полное уничтожение сервера, в том числе уничтожение локальных резервных копий из-за пожара или других проблем.
Вы можете использовать архивы, основанные ленточном резервном копировании всего сервера и перемещать ленты за пределы серверной.
Каждая лента устройство может хранить несколько файлов резервных копий. Ленточные резервные файлы создаются с помощью cpio, tar, dd, и тп. Тем не менее, накопитель на магнитной ленте может быть открыт и закрыт на запись данных различными программами. Вы можете хранить резервные копии на физической ленте. Между каждым файлом ленты есть «файл знак лента». Это используется для того, что бы указать где заканчивается один файл и начинается следующий. Используйте команду mt для позиционирования ленты (перемотки вперед или назад)
Как данные хранятся на ленте
Все файлы хранятся на ленте последовательно в архивах. Первый архив будет находится на физическом начале ленты и так далее.
Именование ленточных устройств в Unix
- /dev/rmt/0 или /dev/rmt/1 или /dev/rmt/63 : Обычное именование лент в Unix. Лента перематывается.
- /dev/rmt/0n : Так называются не перематываемые лентыe т.е. после использования ленты, не оставляет пленку в текущем состоянии для следующей команды.
- /dev/rmt/0b : Использование magtape интерфейс т.е. поведение BSD. Более читаемый различными ОС, такими как AIX, Windows, Linux, FreeBSD, и др.
- /dev/rmt/0l : Сжатие на низком уровне.
- /dev/rmt/0m : Сжатие на среднем уровне.
- /dev/rmt/0u :Сжатие на высоком уровне.
- /dev/rmt/0c : Установлена плотность сжатия.
- /dev/st5 : конкретное имя Linux SCSI ленточного устройства.
- /dev/sa7 : FreeBSD конкретное имя устройства SCSI ленты.
- /dev/esa0 : FreeBSD конкретное имя устройства SCSI ленты, которые могут быть извлечены по окончанию записи.
Как просмотреть установленные SCSI ленточные устройства?
Используйте следующие команды
