PCIe passthrough NVME drives for TrueNAS VM

[wonkyponky]

Hi there,
I have a HP DL380 with an NVME riser card / cage and I already set 3x NVME drives on the Proxmox host itself to provide storage for the VMs. (Host OS sits on RAID1 on the internal hw RAID controller of the machine.)

Now I need additional fileserver storage - my original plan was to passthrough the 5x remaining NVME drives to a VM machine running TrueNAS.

I superficially checked out the guide for PCIe passthrough. With lspci I see each drive with a separate address. And the riser card just expands the PCIe bus so there is no additional controller, etc. involved. So while in theory it should be feasible to passthrough unvirtualized drives I am not sure about:

- can I realiably prevent passthrough of the existing 3x drives? If I mess with them this would kill my virtual machines!
- is it in general a good idea to do it? I would very much prefer the TrueNAS management GUI over setting up ZFS and the sharing features from within the Proxmox host.

Thanks!

 

[wonkyponky]

Hmmm,
is the question clear?
Wouldn't have thought this was such an exotic project ..

 

[UdoB]

I never went this way. You know probably this article? https://www.truenas.com/blog/yes-you-can-virtualize-freenas/

One important sentence: "When creating a TrueNAS VM for production use, the storage controller must be assigned to the VM via PCI passthrough." (Emphases added by me.) Everything else is error prone and may bite you earlier or later.

Repeated disclaimer: I've never created such a system, w/ or w/o passthrough. TrueNAS is best installed on hardware...

Good luck!

 

[wonkyponky]

Hi there,
thanks, well, yes passthrough mandatory, no virtualized drives.

But the question remains - as these are NVMe drives there is no real controller - each drive should act as a separate device. I am not 100% about this though, especially how to safeguard the 3 drives I use on the PVME host, seems like more research should be done about passhtrough. Maybe I'll ask over there with them ZFS cracks.