PBS Backup to TrueNAS: How to do best?

[fchk]

Hello world;

My setup:
- some PVE servers
- a TrueNAS 25.04.2.3 server with 12*12TB of ZFS storage
- PBS 4.0.14 running in a VM on the TrueNAS server

Goal:
- Backup the PVE servers onto a dataset on the TrueNAS server

Question: How? A datastore can only be a local path in the filesystem. Am I supposed to create a NFS mount? What is the best way?

 

[UdoB]

Question: How? A datastore can only be a local path in the filesystem. Am I supposed to create a NFS mount? What is the best way?

Well..., you should be able to create a new zvol (not a "dataset" for files, but a block device) on the primary TrueNAS side. Then attach that device to the PBS VM. You have reached this goal when a new "sdb" (or equivalent) is visible inside the guest.

Then you can format it with a "normal" filesystem (ext2/ext4), mount it and use it a the PBS datastore. All of these steps are offered in the PBS WebGui.

Disclaimer: I do not use this approach - and with the move to SCALE some things may work diferently in TrueNAS than I have in my mind...

 

[news]

I expect 12*12TB of ZFS storage is only hdd based, then use also vdev zfs als special device.
Example vdev0 zfs mirror 2x hdd, vdev1 zfs mirror 2x hdd and vdev2 zfs special device mirror 2x ssd.

Example1:
if you scale up your zfs dataset like this, then the redundancy of your special device.

Example2:
vdev0 zfs raidz2 4x hdd, vdev1 zfs raidz2 4x hdd and vdev2 zfs special device mirror 3x ssd.
2 drives can be lost, without data lost.
If you loose your zfs special device, then you use all of your data in the zfs pool.

 

[Johannes S]

A zvolume (which is otherwise is the best option in terms of performance to go with a PBS VM on TrueNAS) doesn't profit that much from a special device as a dataset:
https://forum.proxmox.com/threads/does-a-zvol-benefit-from-a-metadata-special-device.106292/
A NFS mount would be possible (and obviouvsly could be combined with a special device) but the network file system generates additional overhead ( https://forum.proxmox.com/threads/datastore-performance-tester-for-pbs.148694/ ) which kill performance and HDD isn't a great begin to start with.

How is the layout of your HDD pool? Is it a RAIDZ, a mirror or a striped mirror? Performancewise a striped mirror aka RAID10 together with a special device would be best as long as you can't have an dedicated SSD-only datastore.

Your options are:

• Best option would be to buy a couple of used server ssds with powerloss protection. Two of them would form a new zfs pool in a mirrored setup just for usage as VM zvolume for a datastore in your PBS VM. If you have enough budget you could buy some more so your HDD based pool could have a special device (so it could be used as cold storage for your PBS and you gain better performance with your remaining applications/usecases)
• Add a special device to your HDD pool, then
  • Create a dataset and mount it as NFS to your PBS. Would need some testing, whether this is sufficient for your performance needs-
  • TrueNAS introduced experimental LXC support which allows using a dedicated dataset for a PBS without the overhead of NFS or the limitations of a zvolume: https://forum.proxmox.com/threads/pbs-on-truenas-have-your-cake-and-eat-it-too.162860/ This should give better performance than with NFS (but worse than a zvolume on a SSD) but at the moment this is an experimental feature which might be removed or change in a breaking way at some update

In any case: For existing data an added special device doesn't change anything thus you need to rewrite the data with zfs send/receive, zfs-rewrite (https://openzfs.github.io/openzfs-docs/man/master/8/zfs-rewrite.8.html ) or https://github.com/markusressel/zfs-inplace-rebalancing

 

[UdoB]

Example vdev0 zfs mirror 2x hdd, vdev1 zfs mirror 2x hdd and vdev2 zfs special device mirror 2x ssd.

I was under the impression that his TrueNAS exists already...

 

[UdoB]

A zvolume (which is otherwise is the best option in terms of performance to go with a PBS VM on TrueNAS) doesn't profit that much from a special device

Well..., it absorbs the metadata, doesn't it?

The linked thread is too long... ;-)

...

From #3 over there: "metadata doesn't (only) mean file metadata in this case. zvols are also made up of metadata nodes and data nodes, and the metadata nodes do get stored on a special vdev as well."

 

[Johannes S]

From #3 over there: "metadata doesn't (only) mean file metadata in this case. zvols are also made up of metadata nodes and data nodes, and the metadata nodes do get stored on a special vdev as well."

Yes but as explained in #3 it doesn't achieve the same speedup as with a dataset.

Post in thread 'Does a zvol benefit from a Metadata Special device?'

Mar 11, 2022

metadata doesn't (only) mean file metadata in this case. zvols are also made up of metadata nodes and data nodes, and the metadata nodes do get stored on a special vdev as well. you'll likely see less speedup than with regular datasets, since for filesystems accessing just the metadata is a frequent occurence (things like directory listings, stat-ing paths, reading/writing small files completely embedded the metadata without any data nodes, ...), whereas for zvols all of that happens one layer below inside the VM. if your RAM is adequately sized, most of the metadata should be...

• fabian

• 

 

[Blonov]

Hello world;

My setup:
- some PVE servers
- a TrueNAS 25.04.2.3 server with 12*12TB of ZFS storage
- PBS 4.0.14 running in a VM on the TrueNAS server

Goal:
- Backup the PVE servers onto a dataset on the TrueNAS server

Question: How? A datastore can only be a local path in the filesystem. Am I supposed to create a NFS mount? What is the best way?

How about an iSCSI target on your TrueNAS ?

Phil

 

[Johannes S]

How about an iSCSI target on your TrueNAS ?

This would also be a zfs volume but with ISCSI as additional overhead. There is no need for ISCSI if the VM is also hosted on the TrueNAS and the datastore is already a zfs volume.

 

[Blonov]

This would also be a zfs volume but with ISCSI as additional overhead. There is no need for ISCSI if the VM is also hosted on the TrueNAS and the datastore is already a zfs volume.

Oops. I missed the part where the PBS was hosted on the TrueNAS...

 

[VictorSTS]

If backups are what you value most, install PBS on bare metal following best practices (special device, care with RAIDz depending on performance needed, etc). Leave some room for a TrueNAS VM (or OMV or any other appliance) if you really need file sharing services running on that same hardware.

 

[Zappy]

If backups are what you value most, install PBS on bare metal following best practices (special device, care with RAIDz depending on performance needed, etc). Leave some room for a TrueNAS VM (or OMV or any other appliance) if you really need file sharing services running on that same hardware.

Agree, Of course that's the dream.... but sometimes you have to found a balance .....

the best option in OP situation is vdev , am i correct ?