File Server: pci passthrough to VM and back by iSCSI/NFS?

[Tobias.F]

Hello,

I recently migrated my home lab from Hyper-V to PVE. As next I want to re-design my file server by taking advantage of ZFS file system.

Before, on Hyper-V I used MS Windows Server as File Server that was running as a VM on the Host. On the Host was set-up Storages Spaces with ReFS and the data disc for the file server was a .vmdx file with ReFS file system inside. The backup server was running directly on the host and was storing its data on the same storage space. A classic virtualization. I could make use of the one storage pool by the backup server and by the VM. The disk performance was OK.

After migration to PVE I still have the same set-up. The ZFS pool managed by PVE. The windows file server is running as VM and the data disk is provided as vdev to the VM. On pve host there also is running PBS as backup server, using a directory for the data store on the same ZFS pool. I want to move away from Windows to either True NAS Scale or Open Media Vault. But I believe my following question would also be valid if I stay with Windows as file server.

The key question for me is how I make use of this one ZFS pool by the virtualized file server and by the backup server in the best way?
Possibility 1 (my current set-up)
- PVE is managing the ZFS pool (HDD mirror)
- File Server VM gets disc as vdev
- PBS on PVE can directly access a directory on the ZFS pool

Possibility 2
- PVE is doing PCI passthrough of the SAS controller to the file server VM
- True NAS or OMV is managing the ZFS pool (HDD mirror)
- PBS on PVE get access to the storage by iSCSI or NFS.

Both will work. Any experience or recommendation?

 

[leesteken]

If PVE manages the ZFS pool, can't you run TrueNAS Scale or OpenMediaVault in a container (as they are based on Linux) and pass the storage as a mount point directly? PBS can also run in a (separate) container.
Passthrough comes with its own set of caveats (and ballooning work work) and iSCSI or NFS for local VMs seems like a quite a bit of overhead to me

 

[Tobias.F]

If PVE manages the ZFS pool, can't you run TrueNAS Scale or OpenMediaVault in a container (as they are based on Linux) and pass the storage as a mount point directly? PBS can also run in a (separate) container.
Passthrough comes with its own set of caveats (and ballooning work work) and iSCSI or NFS for local VMs seems like a quite a bit of overhead to me

Yes, a container with mount points would be a 3rd option the the once I listed above. But I am not aware about a supported Installation method for for TrueNAs or OMV in LXC. Any "hack" or self-made installation script that relays on the good will of a single person I would like to avoid. My Windows file server is running since close to a decade. It originally was set-up as Windows 2012R2 under ESXi, was later migrated to Hyper-V and meanwhile is running as Windows 2022 unter PVE without re-installation. This type of "long term robustness" I have in my mind when setting up a system.

Yes, there is overhead. But beside the mount point solution, both options above do have overhead. Either a virtualized disk in the one direction or a iSCSI/NFS target in the other direction.