Can you mount an SMB share as a datastore and then use it to back up directly on tape?

[turin231]

Hello.

We have a new tape ML3 library and we want data that are network shared through SMB to be backed up on tape. The typical back up management suites with the library are a disaster. So i was exploring using PBS with it.

It is my understanding that a tape backup in PSB cannot only be done on a datastore. The issue is that the host of the tape storage that PSB would be installed cannot have enough local storage for the data to be place on a local datastore and then backed up on the tape.

Does it make sense to just mount the smb share on PBS and then make a datastore from the mounted directory to then back up on the tape?
Is there any danger for the data using a solution like this? Is that even possible to do on the PBS?

I know probably performance will be bad but that is not issue. The back up would only be updated once a month and the only reason for recovery would be an actual disaster.

Thank you

 

[AmateurProgrammer]

I have personally tried to do a datastore on an ltfs mountpoint. I does work indeed when backing up data as the backup operations are mostly write only and tape is sequential by nature. However restoring anything from the LTFS would most likely be a nightmare as pbs does not have native support for such datastore and it cannot read the chunks in LTFS order aware way. This would most likely result into insane seeking when trying to restore stuff...

 

[turin231]

I have personally tried to do a datastore on an ltfs mountpoint. I does work indeed when backing up data as the backup operations are mostly write only and tape is sequential by nature. However restoring anything from the LTFS would most likely be a nightmare as pbs does not have native support for such datastore and it cannot read the chunks in LTFS order aware way. This would most likely result into insane seeking when trying to restore stuff...

Any possible recovery would be to recover the whole set anyway and not part of it in our case, so i would not be too worried about the recovery time. That being said would that we a problem of the LTFS due to being linear that would not have an issue with restoring on different file systems?

 

[AmateurProgrammer]

Any possible recovery would be to recover the whole set anyway and not part of it in our case, so i would not be too worried about the recovery time. That being said would that we a problem of the LTFS due to being linear that would not have an issue with restoring on different file systems?

The biggest problem with this is that the reading on chunks from the datastore is not quaranteed to sequential in any way afaik. If the read operations are not sequential, it will result in lots of seeking. PBS does not store the backups in large single files but rather in small chunk files and it uses list to assemble the backups from those chunks. When restoring, PBS will read the relevant chunk files and there is no quarantee of the order in which this happens. Someone can correct me if I'm wrong.

You could however try this. Here are instructions for installing the LTFS fuse software from official repo (https://github.com/LinearTapeFileSystem/ltfs):

apt update && apt install wget build-essential libfuse3-dev automake autoconf libtool icu-devtools libxml2-dev uuid-dev libsnmp-dev python3-xattr

wget https://github.com/LinearTapeFileSystem/ltfs/archive/refs/tags/v2.4.7.1-10515.tar.gz
tar -xvzf v2.4.7.1-10515.tar.gz
cd 'ltfs-2.4.7.1-10515'

./autogen.sh
./configure
make
make install

ldconfig -v

After this you can format a tape as LTFS with mkltfs -d /dev/tape/by-id/...
and mount with ltfs -o sync_type=time@60 -o noatime -o devname=$drive $mountpoint
It currently seems that when mounting, you have to give the drive serial, the /dev/sg and /dev/tape/by-id/.. won't work for some reason...
You can get those available drive serials/id:s with ltfs -o device_list

After this, you can just create datastore normally using that mountpoint.

It would be nice to have native support for this kind of LTFS aware removable datastore but I understand that this is probably kinda niche usecase.

 

[AmateurProgrammer]

I would HEAVILY suggest against using the LTFS mount through SMB protocol. I have had serious problems with the smb server hanging when tape has to be seeked.

 

[turin231]

The biggest problem with this is that the reading on chunks from the datastore is not quaranteed to sequential in any way afaik. If the read operations are not sequential, it will result in lots of seeking. PBS does not store the backups in large single files but rather in small chunk files and it uses list to assemble the backups from those chunks. When restoring, PBS will read the relevant chunk files and there is no quarantee of the order in which this happens. Someone can correct me if I'm wrong.

You could however try this. Here are instructions for installing the LTFS fuse software from official repo (https://github.com/LinearTapeFileSystem/ltfs):

apt update && apt install wget build-essential libfuse3-dev automake autoconf libtool icu-devtools libxml2-dev uuid-dev libsnmp-dev python3-xattr

wget https://github.com/LinearTapeFileSystem/ltfs/archive/refs/tags/v2.4.7.1-10515.tar.gz
tar -xvzf v2.4.7.1-10515.tar.gz
cd 'ltfs-2.4.7.1-10515'

./autogen.sh
./configure
make
make install

ldconfig -v

After this you can format a tape as LTFS with mkltfs -d /dev/tape/by-id/...
and mount with ltfs -o sync_type=time@60 -o noatime -o devname=$drive $mountpoint
It currently seems that when mounting, you have to give the drive serial, the /dev/sg and /dev/tape/by-id/.. won't work for some reason...
You can get those available drive serials/id:s with ltfs -o device_list

After this, you can just create datastore normally using that mountpoint.

It would be nice to have native support for this kind of LTFS aware removable datastore but I understand that this is probably kinda niche usecase.

Wait you are suggesting to mount the tape and then use as a datastore. I was thinking of the opposite. Mount the smb share on the backup server and set it as a datastore path and then back the datastore to the tape. Is that possible?

 

[AmateurProgrammer]

Wait you are suggesting to mount the tape and then use as a datastore. I was thinking of the opposite. Mount the smb share on the backup server and set it as a datastore path and then back the datastore to the tape. Is that possible?

No, I cannot recommend using LTFS as a datastore. Especially not with SMB protocol for the reasons stated previously. Or did you mean a separate server with disk that is being shared through smb and that is used by the pbs? Of course you can try and report your findings. It would be interesting to see how severe the impact of seeking would be for the restore operation when there is something like 1tb of data on the tape.

 

[Johannes S]

Wait you are suggesting to mount the tape and then use as a datastore. I was thinking of the opposite. Mount the smb share on the backup server and set it as a datastore path and then back the datastore to the tape. Is that possible?

Do you want to do the tape backup with PBS or with a different tool (e.G. tar)? I would only use PBS native tape support

 

[turin231]

Do you want to do the tape backup with PBS or with a different tool (e.G. tar)? I would only use PBS native tape support

I want to do the back up with PBS. The data is on an SMB network share on a different device than the PBS and the PBS device does not have enough local space to store all the data that need to be backed up. It is only connected to the Tape library. I want a way to bring the data from the SMB share to the tape.

 

[turin231]

No, I cannot recommend using LTFS as a datastore. Especially not with SMB protocol for the reasons stated previously. Or did you mean a separate server with disk that is being shared through smb and that is used by the pbs? Of course you can try and report your findings. It would be interesting to see how severe the impact of seeking would be for the restore operation when there is something like 1tb of data on the tape.

See above reply