Can I automate tar’s multi-volume-feature?
Ok, so I’ve just had a read through this page after a way to improve my current backup solution on my Debian server. Tar seems to be offering a quite nice multi-volume feature, although when I try it out, it asks me to Prepare volume #X for ‘mybackup.tar.gz’ and hit return: . How should I automate this as I would like to take usage of this feature in an automated CRON script where no one is there to push return and enter whatever is rquired by the multi-volume prompt. Is using split the only way?
2 Answers 2
printf 'n file-%02d.tar\n' | tar -ML 716800 -cf file-01.tar Documents/ 2>/dev/null where 100 is a number greater or equal to the number of volumes.
Setting a big number should not be a problem, though I tend to not take a ridiculous one.
An alternative could be a «next volume» script, that you can set with the -F option,
tar -ML 716800 -F './myscript file' -cf file.tar Documents/ 2>/dev/null #!/bin/bash prefix="$1" n=1 while [[ -e "$prefix-$n.tar" ]]; do ((n++)) done mv "$prefix.tar" "$prefix-$n.tar" echo "$prefix-$n.tar" It will be executed at each volume end, and will move file.tar to the appropriate fileNNN.tar . For the last volume the script will not be executed, so the last volume name stay file.tar .
I ended up with the following elaborated solution.
Here are two script, one for the creation and the other for the extraction:
#!/bin/bash # CREATION SCRIPT # save on file the initial volume number echo 1 >number # multi-volume archive creation tar -ML 100000 -F './tar-multi-volume-script c file' -cf file.tar Documents2/ 2>&- # execute the "change-volume" script a last time ./tar-multi-volume-script c file #!/bin/bash # EXTRACTION SCRIPT # save on file the initial volume number echo 1 >number # execute the "change-volume" script a first time ./tar-multi-volume-script x file # multi-volume archive extraction tar -M -F './tar-multi-volume-script x file' -xf file.tar 2>&- # remove a spurious file rm file.tar where ./tar-multi-volume-script is given by
#!/bin/bash # TAR INVOKED SCRIPT mode="$1" prefix="$2" n=$(number Obviously you have to change many bits here and there to adapt to your situation and to be sure it would work in cron , that is always a little challenge.
Creating and extracting multivolume tar files
Some servers do not support files larger than 2GB, also when creating archives larger than 2GB most probably server will run out of memory or other resources. So is best to split large archives in multiple files. This is a good way to move large sites to another server.
Creating the multivolume tar file
If for example you have a folder «my_documents» and you want to create multivolume tar archive with its contents, this command will start archiving it and will split the files to 1.2GB:
tar -cML 1258291 -f my_documents.tar my_documents
When first file reached 1.2GB (after some minutes) it will display a prompt to move to next file, it will ask if volume #2 for this archive is ready, type this answer, notice the «2«:
press enter, it will ask if files are ready, press enter again to confirm without typing anything. On next prompt type «n my_documents3.tar» and press enter twice again, and so on. You should now have multiple files of maximum 1.2GB named: my_documents.tar, my_documents2.tar, my_documents3.tar.
Unlike with zip files, it doesn’t seem to be a command for testing multivolume tar files.
If you created the files in order to move a large site, I would recommend to move them with wget triggered from destination server:
wget http://old_site.com/my_documents.tar
You should do some basic testing to compare file integrity if copied from another server, for example use «cksum my_documents.tar» on old and new server to make sure it returns same result, meaning that files are the same.
To unpack the multivolume tar archive
Use this command to unpack the multivolume tar archive, this would probably hapen on the new server if you moved a site:
After some work (minutes on most servers), it will ask for next file, type this:
Then on next question type n my_documents3.tar, etc.
Creating single tar files:
To create one single tar file, for example archiving «images» folder:
unpack single files (not multivolume) with this command:
Multi volume archive linux
To create an archive that is larger than will fit on a single unit of the media, use the —multi-volume ( -M ) option in conjunction with the —create option (see create). A archive can be manipulated like any other archive (provided the —multi-volume option is specified), but is stored on more than one tape or disk.
When you specify —multi-volume , tar does not report an error when it comes to the end of an archive volume (when reading), or the end of the media (when writing). Instead, it prompts you to load a new storage volume. If the archive is on a magnetic tape, you should change tapes when you see the prompt; if the archive is on a floppy disk, you should change disks; etc.
You can read each individual volume of a multi-volume archive as if it were an archive by itself. For example, to list the contents of one volume, use —list , without —multi-volume specified. To extract an archive member from one volume (assuming it is described that volume), use —extract , again without —multi-volume .
If an archive member is split across volumes (ie. its entry begins on one volume of the media and ends on another), you need to specify —multi-volume to extract it successfully. In this case, you should load the volume where the archive member starts, and use ‘ tar —extract —multi-volume ’— tar will prompt for later volumes as it needs them. See extracting archives, for more information about extracting archives.
—info-script= script-name ( —new-volume-script= script-name , -F script-name ) (see info-script) is like —multi-volume ( -M ), except that tar does not prompt you directly to change media volumes when a volume is full—instead, tar runs commands you have stored in script-name . For example, this option can be used to eject cassettes, or to broadcast messages such as ‘ Someone please come change my tape ’ when performing unattended backups. When script-name is done, tar will assume that the media has been changed.
Multi-volume archives can be modified like any other archive. To add files to a multi-volume archive, you need to only mount the last volume of the archive media (and new volumes, if needed). For all other operations, you need to use the entire archive.
If a multi-volume archive was labeled using —label= archive-label ( -V archive-label ) (see label) when it was created, tar will not automatically label volumes which are added later. To label subsequent volumes, specify —label= archive-label again in conjunction with the —append , —update or —concatenate operation.
—multi-volume -M Creates a multi-volume archive, when used in conjunction with —create ( -c ). To perform any other operation on a multi-volume archive, specify —multi-volume in conjunction with that operation.
—info-script= program-file —new-volume-script= program-file -F program-file Creates a multi-volume archive via a script. Used in conjunction with —create ( -c ). See info-script, dor a detailed discussion.
Beware that there is no real standard about the proper way, for a tar archive, to span volume boundaries. If you have a multi-volume created by some vendor’s tar , there is almost no chance you could read all the volumes with GNU tar . The converse is also true: you may not expect multi-volume archives created by GNU tar to be fully recovered by vendor’s tar . Since there is little chance that, in mixed system configurations, some vendor’s tar will work on another vendor’s machine, and there is a great chance that GNU tar will work on most of them, your best bet is to install GNU tar on all machines between which you know exchange of files is possible.