LTO-5 Tape Drive as backup drive

[tafkaz]

Hi!
I have an LTO-5 SAS Tapedrive here that i would like to use for daily snapshot backups.
Only i don't really know how to enbind it correctly into a proxmox ve 2.1 host.
Any ideas?

Thank you
Sascha

 

[udo]

Hi!
I have an LTO-5 SAS Tapedrive here that i would like to use for daily snapshot backups.
Only i don't really know how to enbind it correctly into a proxmox ve 2.1 host.
Any ideas?

Thank you
Sascha

Hi Sascha,
with an backup-software! I use for this case bacula (my backup-server is an openvz-vm with two lto-4 drives).
If you backup also network clients use an fast spooldisk.

Udo

 

[tafkaz]

Oh i see,
i thought i could just add it as an iSCSI-Storage to Proxmox VE and then backup to it directly... But that is not possible then?
I will have a look at bacula then, any other hints, what software i could use?
Basically i would just want to copy everydays vzdumps to the tapes then.
Thanks again
Sascha

 

[udo]

Oh i see,
i thought i could just add it as an iSCSI-Storage to Proxmox VE and then backup to it directly... But that is not possible then?
I will have a look at bacula then, any other hints, what software i could use?
Basically i would just want to copy everydays vzdumps to the tapes then.
Thanks again
Sascha

Hi,
you can also copy the files directly to the tape with tar or cpio or so. But in this case you must do the cartrige-change by yourself (which the same with an backup-software, because you have only a drive and not a jukebox) and you must use the right device (norewind) otherwise you allways overwritten the content. And you must remember the content of the tapes by yourself.
See "man tar" and "man mt".

Udo

 

[tafkaz]

So i tried to copy my vzdump-snapshots to tape with:
#!/bin/bash

TAPE=/dev/st0
echo "Start Time: `date +'%Y-%m-%d %T'`"
mt -f $TAPE defcompression 1
mt -f $TAPE rewind
cd /usr/local/backup/daily/dump/
tar -cf $TAPE ./vzdump*.gz
echo "Start writing Tape:`date +'%Y-%m-%d %T'`."
echo "Stop writing Tape:`date +'%Y-%m-%d %T'`"
mt -f $TAPE eject
echo "Backup finished: `date +'%Y-%m-%d %T'`"


unfortunately after about an hour or so i get an input/output error.
in syslog i see:
Oct 10 10:17:34 proxmox1 kernel: st0: Sense Key : Aborted Command [current]
Oct 10 10:17:34 proxmox1 kernel: st0: Add. Sense: Tagged overlapped commands (task tag 0)

Is there some kind of filesize-limit i am facing? the vzdump file is about 250 GB big.
And if so, how can i backup these files to tape then? Lto-5 media should be big enough.

Thanx again
Sascha

 

[udo]

So i tried to copy my vzdump-snapshots to tape with:
#!/bin/bash

TAPE=/dev/st0
echo "Start Time: `date +'%Y-%m-%d %T'`"
mt -f $TAPE defcompression 1
mt -f $TAPE rewind
cd /usr/local/backup/daily/dump/
tar -cf $TAPE ./vzdump*.gz
echo "Start writing Tape:`date +'%Y-%m-%d %T'`."
echo "Stop writing Tape:`date +'%Y-%m-%d %T'`"
mt -f $TAPE eject
echo "Backup finished: `date +'%Y-%m-%d %T'`"


unfortunately after about an hour or so i get an input/output error.
in syslog i see:
Oct 10 10:17:34 proxmox1 kernel: st0: Sense Key : Aborted Command [current]
Oct 10 10:17:34 proxmox1 kernel: st0: Add. Sense: Tagged overlapped commands (task tag 0)

Is there some kind of filesize-limit i am facing? the vzdump file is about 250 GB big.
And if so, how can i backup these files to tape then? Lto-5 media should be big enough.

Thanx again
Sascha

Hi,
two things:
I would suggest to use the norewind-device (to not overwrite the first backup during another tar - you can clean used tapes with mt and also jump to the writen end of an tape): /dev/nst0
You should use an bigger blocksize with tar (--blocksize=256) and you can look how much files are copied with the v-flag.

Udo

 

[tafkaz]

Ok we're getting somewhere here!

i thought I'd rather user st0 instead of nst0 just because it'll overwriter the last file.
So the tape should never get to full with data, that was the idea.
The customer will then change the tape everyday, having 7 tapes in total.

With
tar -b 256 -cvf $TAPE ./vzdump*.gz
it seems to copy the files to the tape, no input/output errors anymore.

But:
tar -tzf /dev/st0 shows:

tar (child): /dev/st0: Cannot read: Cannot allocate memory
tar (child): At beginning of tape, quitting now
tar (child): Error is not recoverable: exiting now

gzip: stdin: unexpected end of file
tar: Child returned status 2
tar: Error is not recoverable: exiting now

in syslog:
Oct 10 13:10:17 proxmox1 kernel: st0: Failed to read 131072 byte block with 10240 byte transfer


thanks again
Sascha

 

[udo]

Ok we're getting somewhere here!

...
With
tar -b 256 -cvf $TAPE ./vzdump*.gz
it seems to copy the files to the tape, no input/output errors anymore.

But:
tar -tzf /dev/st0 shows:

tar (child): /dev/st0: Cannot read: Cannot allocate memory
tar (child): At beginning of tape, quitting now
tar (child): Error is not recoverable: exiting now
...

Hi Sascha,
the z-flag is for compression (gzip) but you don't use compression (only from lto-drive, which is transparent) to write the data.

Use simply "tar tvf /dev/nst0" or "tar xvf /dev/nst0" or st0.

Udo

 

[tafkaz]

Hi Udo,
many thanks for your help!
Unfortunately:

tar -tvf /dev/st0
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: At beginning of tape, quitting now
tar: Error is not recoverable: exiting now

Oct 10 15:01:51 proxmox1 kernel: st0: Failed to read 131072 byte block with 10240 byte transfer.

 

[udo]

Hi Udo,
many thanks for your help!
Unfortunately:

tar -tvf /dev/st0
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: At beginning of tape, quitting now
tar: Error is not recoverable: exiting now

Oct 10 15:01:51 proxmox1 kernel: st0: Failed to read 131072 byte block with 10240 byte transfer.

Hi,
I guess you are not at the right tape-position.
Try

mt -f /dev/nst0 rewind
tar tvf /dev/nst0
# perhaps again "tar tvf /dev/nst0" and use nst0 because other way the tape is spooling back again
# or
mt -f /dev/nst0 fsf 1
tar tvf /dev/nst0

Udo

 

[tafkaz]

Not yet, sorry:

mt -f /dev/nst0 fsf 1
tar tvf /dev/nst0

tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors

Is there a way to somply reset the whole tape, and start over?
Maybe that is the problem?

Thanks again
Sascha

 

[udo]

Not yet, sorry:

mt -f /dev/nst0 fsf 1
tar tvf /dev/nst0

tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors

Is there a way to somply reset the whole tape, and start over?
Maybe that is the problem?

Thanks again
Sascha

Hi,

mt -f /dev/st0 erase

should erase the tape (never tried with lto).

BTW. the handling with mt and such on is an reason why many people like backup-software

Udo

 

[tafkaz]

Hi Udo,
I tried with
mt -f /dev/nst0 weof
for now, which is a bit faster!

Now copying data again...if that doesn't help, i will give full erase a try!

Wich Backup software would you hint me to under Linux / Debian?
I find bacula a little overweight for what i want...
But of course some backup software that is able to copy, let's say, three instances then alway remove the oldest....that would be kinda posh...
Any recommendation for some easy to configure backup software?

Thank you very much
Sascha

 

[tafkaz]

No, still not working...
Are you sure that taring with -b 256 is the right thing to do? Maybe that's the problem:



tar -tvf /dev/nst0
tar: /dev/nst0: Cannot read: Cannot allocate memory
tar: At beginning of tape, quitting now
tar: Error is not recoverable: exiting now


then i tried:
tar -b 256 -tvf /dev/nst0
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Archive contains `[B\017\316\303nF1Nc\222\020' where numeric off_t value expected
tar: Archive contains `\332r\275I\245\215UT' where numeric mode_t value expected
tar: Archive contains `1,O\0\216?\375t\354Þ¨' where numeric time_t value expected
tar: Archive contains `\025R~o\231\034\363Y' where numeric uid_t value expected
tar: Archive contains `]-\316jn\247\330\350' where numeric gid_t value expected
tar: Archive contains `\025R~o\231\034\363Y' where numeric uid_t value expected
tar: Archive contains `]-\316jn\247\330\350' where numeric gid_t value expected


strange....
Any idea?
Thanks Sascha

 

[tafkaz]

Hi!
Doesn't work...
after erase, still the same problem...

once again...this is what i do:

#!/bin/bash

TAPE=/dev/st0
echo "Start Time: `date +'%Y-%m-%d %T'`"
mt -f $TAPE defcompression 1
mt -f $TAPE rewind
cd /usr/local/backup/daily/dump/
tar -b 256 -cvf $TAPE ./vzdump*.gz
echo "Start writing Tape:`date +'%Y-%m-%d %T'`."
echo "Stop writing Tape:`date +'%Y-%m-%d %T'`"
mt -f $TAPE eject
echo "Backup finished: `date +'%Y-%m-%d %T'`"

it seems to copy the file to tape, but then, there is no way, i can see the file on the tape:

mt -f /dev/st0 load
mt -f /dev/st0 rewind
tar -tvf /dev/st0

tar: /dev/st0: Cannot read: Cannot allocate memory
tar: At beginning of tape, quitting now
tar: Error is not recoverable: exiting now

Oct 11 10:25:21 proxmox1 kernel: st0: Failed to read 131072 byte block with 10240 byte transfer.


Any idea, what i am doing wrong here?
The file i am trying to tar to the tape is
vzdump-qemu-1002-2012_10_10-20_00_02.tar.gz
and it is 250G big.

Please help, i am really stuck here.
Thank you very much for your assistance
Sascha




EDIT:
Seems to work now, need to put it tar -b 256 -tvf /dev/st0 of course! :-D

But another question...
Could i change my skript to make it copy e.g. 3 files and from then on always delete the oldest one before copying the next one?
Would be cool if i had more than one dump on tape!

Thanks
Sascha

 

[tafkaz]

But another question...
Could i change my skript to make it copy e.g. 3 files and from then on always delete the oldest one before copying the next one?
Would be cool if i had more than one dump on tape!

Thanks
Sascha

Hi,
my backups work pretty fine so far, but i was still wondering, if there is a way to hold more than one backup file before deleting the oldest backup.
If anyone has an idea, i would love to try.
Also, as there's not enough space on my Harddisk-RAID to store more than one backup, it wouldn't be too bad to start backuping directly to tape instead of copying the disk-backuped file to tape as i am doing now.
Ideas very welcome as well...

thank you
Sascha

 

[frost_online]

I don't think there is a way to backup data directly to the tapes and if there is , i wont suggest that. As backing up data on hard disk and then moving to tape is more reliable and can be checked easily.

 

[tafkaz]

ok, i see, but is there any way to hold more than one backup file on the tape before deleting the oldest backup?

 

[tom]

yes. you can define the number of backups via the storage definition.

 

[tafkaz]

Hi!
Yes i know, but as there's not enough space on my local RAID to store more than one backup simultanously, i was thinking of a possibility to store more than one backup file on the tape.

This being my backup to tape script

#!/bin/bash

TAPE=/dev/st0
echo "Start Time: `date +'%Y-%m-%d %T'`"
mt -f $TAPE defcompression 1
mt -f $TAPE rewind
cd /usr/local/backup/daily/dump/
tar -b 256 -cvf $TAPE ./vzdump*.gz
echo "Start writing Tape:`date +'%Y-%m-%d %T'`."
echo "Stop writing Tape:`date +'%Y-%m-%d %T'`"
mt -f $TAPE eject
echo "Backup finished: `date +'%Y-%m-%d %T'`"

What could i do to leave 2 or 3 backups on the tape before deleting the oldest one?

Thanks
Sascha