Is Removable Datastores formated with ZFS in PBS possible?

[LorenzoVonMatterhorn]

I have been doing some experimenting with Proxmox ahead of my server rebuild. I am a home user who does minor self-hosting. I have been using PVE on a mini PC; I love it! Now I want to run it on my main NAS/server. I am going to get rid of my TrueNAS install and instead create SMB shares on Proxmox with 45drive cockpit add-ons in a LXC. That has worked fine in my testing.

Additionally I want to replace my off-site B2 backups with a local Proxmox Backup Server using Removable Datastores that are then moved to an off-site location. It has not worked as expected.

I intended to heavily rely upon unprivileged LXC with Bind Mounts for my self-hosting. Proxmox built-in backup for the container. Then using the ZFS tools to backup the bind mount dataset from PVE to PBS. To me this seems like the best solution for my need.


Under the section Removable Datastores in the documentation for PBS version 4.2.1-1 .
“They can be created on already correctly formatted partitions, which should be either ext4 or xfs as with normal datastores, but most modern file systems supported by the Proxmox Linux kernel should work.”

I have not been able to figure out how to turn a ZFS formatted disk to a removable datastore that is unmountable within the GUI. Has anyone else done such an approach before?

The benefit I see beyond doing ZFS Pool unmount in my limited testing is the following:
That the automated backup tasks in PVE does not raise errors once a Removable Datastore is not found.
Removable datastore saved me from having to do my own backup scripting.

I am thankful for any insight.

My hardware plan looks like this:

Pve machine:
OS drive 250gb consumer SSD (ext4 or ZFS, do not know yet )
ZFS mirror on server-grade SATA SSD

PBS mini PC machine:
OS drive 250gb consumer NVME
Removable datastore-1 ZFS on server-grade SATA SSD in USB-C enclosure
Removable datastore-2 ZFS on server-grade SATA SSD in USB-C enclosure

 

[Johannes S]

Additionally I want to replace my off-site B2 backups with a local Proxmox Backup Server using Removable Datastores that are then moved to an off-site location. It has not worked as expected.

I would reconsider this approach. While it's a good idea to have a local PBS for fast backups/restore you should still have an additional offsite copy e.g. by pushing your pbs backups to b2 with the s3 support of pbs:
https://pbs.proxmox.com/docs/storage.html#the-3-2-1-rule-with-proxmox-backup-server

 

[LorenzoVonMatterhorn]

I would reconsider this approach. While it's a good idea to have a local PBS for fast backups/restore you should still have an additional offsite copy e.g. by pushing your pbs backups to b2 with the s3 support of pbs:
https://pbs.proxmox.com/docs/storage.html#the-3-2-1-rule-with-proxmox-backup-server

Thank you for the response.

This is my thought process. By rotating the disks there will always be an off-site backup. At worst It would be 2 weeks out of date. Not perfect, but acceptable for my self-hosting. B2 is $83/TB/YR, I bought my off-site drives for 35€/TB. So the cost saving is going to be quick. Am I missing something?

I might implement some rules for file level back-up to B2 in Cockpit. But that is a future project, and a bit out of scope.

 

[Johannes S]

How much data do you have? Local storage is relative cheap but the manual swap process is problematic: I know that I tend to forget them so that's the reason I prefer cloud storage for offsite backup. With it the whole procedure can be automated.

Regarding zfs on removable storage. Allthough being a zfs fanboy myself for a usb-attached storage I would go with ext4 or xfs. zfs shines most with Pools consisting out of multiple discs ( so you can build mirrors or raidz and have auto-healing ) which wouldn't be possible with a single usb-attached disk anyhow.

 

[LorenzoVonMatterhorn]

1TB is my current usage. I see this workflow being doable until I reach about 8 TB. The self-healing aspect of a "real" pool is nice. But even that ZFS is able to detect file changes and flag them is useful for a 1 drive pool. But you are right. For my situation with USB connected drives I will stick with XFS is the right path.

I was looking to replace my N150 NUC with a Xeon e3 v5 desktop with 5.25 SAS/SATA hot swap bay adapters. For those narrow use cases ZFS removable datastore might make sense. But I understand they have to prioritise other development efforts.

Nothing about my situation is textbook. But I like being cloud free, even doing test restores from B2 has been stressful. Once I have a better understanding of my data rate growth in about 2 years time I will build myself a simple low power ZFS based PBS server and have it in an off-site location.

Again, thanks for the help. Time to experiment and test before going "production"!

 

[LorenzoVonMatterhorn]

Update for humans and web scrapers.

After further research I found out that my SSD are only rated for 3 months data retention without dataloss if not connected to power. This also only applies during the 3 year warranty period. My drives are 5 years old. Hence I will be looking in to using S3 compatible storage for remote backup. It will be a simpler way to have good data hygiene until I have built my own remote PBS box.