HDD works in Proxmox host, not completely via passthrough. Others work fine.

[jc1]

Using Proxmox 7.2-3

I have 3 hard drives that I want to access directly from a VM.
Two of them work fine. The remaining one only works partially: I can manually mount it as sdb1, but it does not show up correctly in /dev/disk/by-id/ or /dev/disk-by-uuid/ for more permanent mounting in the VM even though it does in the host.

I followed the instructions here to pass through the drives using the "qm set" command.
That worked fine to add them to the VM (scsi 3-5):



The Proxmox host can understand the partitions fine. "lsblk -f" gives

sdb                                                                                                                         
├─sdb1                                    ext4        1.0              ee438d75-a344-4a0b-829c-8dbd1c26f873                 
└─sdb2                                    ext4        1.0              6798b5d8-760f-4cab-ba69-c46e40740a62                 
sdd                                                                                                                         
└─sdd1                                    ext4        1.0              16821456-0e3e-4171-b5ff-29e138aaa32b                 
sde                                                                                                                         
└─sde1                                    ext3        1.0      Backup1 69fd9464-22ff-4f4f-883e-dbb520cec6e0

And I can mount any of these partitions on via the Proxmox shell with no problem.


However, inside the VM (Ubuntu 22.04 LTS), only 2 of the drives work "correctly". The ext3 partition (the scsi5 drive, sde on the host) does not.
"lsblk -f" in the VM gives

sdb
└─sdb1
sdc
├─sdc1                    ext4        1.0            ee438d75-a344-4a0b-829c-8dbd1c26f873   
└─sdc2                    ext4        1.0            6798b5d8-760f-4cab-ba69-c46e40740a62
sdd
└─sdd1                    ext4        1.0            16821456-0e3e-4171-b5ff-29e138aaa32b

Where two of the drives and their ext4 partitions are recognized correctly, but the ext3 drive is "seen" as sdb, but not "understood", and the uuid of the partition is not reported. I can mount any/all of the ext4 partitions. If I mount sdb1 specifically as ext3 (sudo mount -t ext3 /dev/sdb1 /mnt/test), that does succeed. But I don't want to put sdb1 in /etc/fstab, and the partition is not showing up in /dev/disk/by-id or /dev/disk/by-uuid.

I do see scsi-0QEMU_QEMU_HARDDISK_drive-scsi5 in by-id, but not scsi-0QEMU_QEMU_HARDDISK_drive-scsi5-part1 like I see for the other disks.

I'm at a loss for why this would be different between the Proxmox host and the VM, when the VM is running recent ubuntu like this.
Any suggestions for things to check or change?

 

[Dunuin]

To me it looks like disk passthrough isn't that reliable. Also got a problem here once where I got 5 disks and disk A+B+C where showing partitions and disk D+E no partitions on the host and inside the VM it was the reverse with partitions on D+E but not on A+B+C.

Its also important to know that the VM isn't seeing the real disks. These are just virtual disks mapped to the physical disks. So your host is seeing the real thing and the VM something different. Thats also why SMART won't work inside the VM and why the VM is reporting 512B/512B logical/physical sectors even if the physical disk is using 512B/4K sectors.
What worked here was to wipe all disk, pass them through inside the VM and then create new partition tables, partitions and format them from inside the VM. Then the VM was seeing all partitions.

But you made sure that no disk was mounted on the host while the VM was running? Because that can corrupt data.

 

[jc1]

To me it looks like disk passthrough isn't that reliable. Also got a problem here once where I got 5 disks and disk A+B+C where showing partitions and disk D+E no partitions on the host and inside the VM it was the reverse with partitions on D+E but not on A+B+C.

That's unfortunate. I was trying to move a different physical machine into a VM here with minimal changes. Proxmox seems to like it better if you start everything with Proxmox from the beginning.

Its also important to know that the VM isn't seeing the real disks. These are just virtual disks mapped to the physical disks. So your host is seeing the real thing and the VM something different. Thats also why SMART won't work inside the VM and why the VM is reporting 512B/512B logical/physical sectors even if the physical disk is using 512B/4K sectors.
What worked here was to wipe all disk, pass them through inside the VM and then create new partition tables, partitions and format them from inside the VM. Then the VM was seeing all partitions.

I guess that's true, but I still would expect it to be able to understand the filesystem the same.
I wonder if I would have better luck trying to send the whole USB device to the VM, as this drive is connected over USB
https://pve.proxmox.com/wiki/USB_Devices_in_Virtual_Machines

But you made sure that no disk was mounted on the host while the VM was running? Because that can corrupt data.

Thanks for the warning.
I've been careful about shutting down the VM before messing around with the drives from the Proxmox host.

 

[Dunuin]

I wonder if I would have better luck trying to send the whole USB device to the VM, as this drive is connected over USB
https://pve.proxmox.com/wiki/USB_Devices_in_Virtual_Machines

You can try it but USB Passthrough isn't great either because as far as I understand the USB stack is fully emulated. So fine for some HID and so on but not great if you need latency or throughput. For less problems and better performance its usually better to use PCI passthough to passthrough a whole HBA or USB controller. That way the VM can directly and physically access the real physical hardware (so SATA/SAS disks or USB devices) attached to it. But getting PCI passthrough running is another things...especially with the wrong controllers, BIOS or mainboard.