Possible to hide disks that have been directly passed to a VM

[CelticWebs]

I've setup a TrueNAS VM on my Proxmox. To allow it to "do its thing", I've passed the disks directly to the VM by their disk IDs with Serial numbers. This works perfectly, Truenas sees the disks and has happily made a ZFS pool out of them, I believe this is the best way to do it where the pool can be monitored by Truenas and warnings be sent if anything starts to degrade.

My issue now is that Proxmox still see those disks as unused under the disk list, I now want to add some more disks in to the server in the spare bays for use with proxmox, The downside is that I know that when I add disks, there's a possibility that after a reboot, the labelling of sda sdb etc is likely to change. This has me concerned that I will accidently setup / format one of the disks that's passed to the TrueNAS VM. Is there a way to hide the disks or just mark them as used so that this can't happen?

I sort of assumed that at worst they woud show as having data on them, allowing me to se they were in use, but they don't, they just show as unused disk.

Any suggestions on how to proceed without risking data?

Thanks!

 

[Dunuin]

I believe this is the best way to do it where the pool can be monitored by Truenas and warnings be sent if anything starts to degrade.

Without PCI passthrough of NVMe SSDs or an HBA there is no working SMART from inside the VM so TrueNAS can't monitor the disks health nor send warnings when a smart selftest fails. Then you need to set up external monitoring/alterting (via CLI) on the PVE host to monitor your disks health.

The downside is that I know that when I add disks, there's a possibility that after a reboot, the labelling of sda sdb etc is likely to change. This has me concerned that I will accidently setup / format one of the disks that's passed to the TrueNAS VM.

Thats why you shouldn't use sda, sdb and so on. Passthrough and mount the disks using /dev/disk/by-id/... and you should be fine. Then it won't matter if those sda/sdb/sdc... change.

Is there a way to hide the disks or just mark them as used so that this can't happen?

Not that I know of.

 

[CelticWebs]

I just realised smart doesn't work as you said. Disk aren't "really" passed through it would appear. I can't pass the whole HBA because some drives are still needed in Proxmox. It's Supermicro Super Chassis with a 36 disk controller / expander in there. So it's not actually going to work as I'd hoped anyway.

I had passed them using IDs, what I was saying was that I currently know those ID related to sda-sdh, but after a reboot adding new disks, that will possibly change, making it difficult to work out which disks in the list are the ones directly passed to TrueNas. I just tested and I can happily format one that via Proxmox that True has is using in a pool.

Looks like a rethink is required!

Thanks for the prompt response.

 

[Dunuin]

Disk aren't "really" passed through it would appear.

Yes, TrueNAS is still working with virtual disks. Only way for the guestOS to directly access the real physical disks is to use PCI passthrough of the whole HBA or individual NVMe disks.

Just don't work with sda/sdb and so on at all. If you need to do something on the CLI use /dev/disk/by-id or UUIDs. And if you need to do something on PVEs webUI don't look at the sda, sdb and so on but at the column with the model and serial. If you always look at the serial you can't wipe the wrong disk, no matter what sdX it is called at the moment.