I am current playing with this. I have three PBS servers running in a Proxmox cluster of 3 machines ( Ryzen 3900x, HP z440, HP z840 ). The PBS machines are as follows:
- HP z440 with local ZFS raidz1 of 3 disks.
- HP z840 with TrueNAS CIFS filesystem
- TrueNAS "instances" VM with ZFS zvol on Virtio-BLK.
I have about 30 VMs that don't change much so the actual writes are minimal.
I expected a lot of differences between the PBSes, but as far as time is concerned they are all about the same. The longest time is the Ryzen system that has 20 VMs and containers which take the following duration: 22:27 , 22:41, 22:11. Each has about 380GB of 'read' data but not a lot of 'write' data since they haven't' changed much.
so are far as time is concerned, I have not seen any benefit. I do not know about the other advantage ( such as zfs deduplication ).
The reason I'm testing is I want to take down the hp z440 box and move the drives to another machine and I don't know whether to use CIFS/NFS or a truenas vm with it's own zvol.
The advantage of CIFS/NFS would be that I can see the files which gives me some confidence that I can reinstall the PBS and use the backups ( well, I have heard this is very doable but I have not actually tried it). I'm actually not sure how to access the Truenas zvol if the PBS goes down ( probably possible but I have not tried it).
BTW, my TrueNAS runs in my HP z840, so the PBS is actually doubly virtualized: HP Z840 --> TruNAS --> PBS. I sort of expected this to be the slowest, but it is marginally faster than the others.
I would like to hear if anybody else sees advantages of ZFS over CIFS/NFS for PBS datastores.
