SATA controller passthrough working only first VM boot

[GenGT]

Hello,

I have a m.2 to pcie JMicron JMB585 SATA adapter with 5 SATA ports. Pcie passthrough is working without any problem the first boot of the VM, the problem comes once the VM is shutdown correctly (inside VM) and I want to start the VM again. All proxmox server hangs, and I have to reset the server manually, the problem is the same if I shutdown the VM, reboot the Server and boot the VM again, it only works again when the system crash and I have to reboot the server manually.

I've followed this topic https://forum.proxmox.com/threads/sata-controller-passthrough.97322/ without success. This is my config:

Spoiler: VM Config

args:
boot: order=sata0
cores: 2
hostpci0: 0000:01:00,pcie=1
machine: q35
memory: 4096
meta: creation-qemu=6.1.0,ctime=1652103298
name: SynologyPill
net0: virtio=xx:xx:xx:xx,bridge=vmbr0
numa: 0
onboot: 0
ostype: l26
sata0: disk2TB:100/vm-100-disk-0.raw,size=1G,ssd=1
scsihw: virtio-scsi-pci
serial0: socket
smbios1: uuid=73f7a62a-b90e-4827-bd47-1afad44b128a
sockets: 1
tablet: 0
vmgenid: 717c56f0-511e-4838-95c3-9e605c183d22

Sata controller is in a single group

IOMMU group 10 01:00.0 SATA controller [0106]: JMicron Technology Corp. JMB58x AHCI SATA controller [197b:0585]
        Subsystem: JMicron Technology Corp. JMB58x AHCI SATA controller [197b:0000]
        Kernel driver in use: vfio-pci
        Kernel modules: ahci

If I add a .conf file in modprobe.d with the following

options vfio_pci ids=197b:0585

Now the problem happens always, booting the VM hangs the server, and the Sata controller drive is changed from vfio-pci to ahci

IOMMU group 10 01:00.0 SATA controller [0106]: JMicron Technology Corp. JMB58x AHCI SATA controller [197b:0585]
        Subsystem: JMicron Technology Corp. JMB58x AHCI SATA controller [197b:0000]
        Kernel driver in use: ahci
        Kernel modules: ahci

Any idea of what is happening?

Regards

 

[leesteken]

Pcie passthrough is working without any problem the first boot of the VM, the problem comes once the VM is shutdown correctly (inside VM) and I want to start the VM again.

Not all PCIe devices do proper Function Level Reset. This is unfortunately not uncommon and prevents such a device from working after the VM is shutdown (until the host is rebooted). Not all such devices take down the Proxmox host, but it does happen with some devices.
You could try work-arounds like virtually unplugging the device and rescanning the PCIe bus (to force a hardware reset). Or try to find someone who knows a work-around for this particular device. Or search of a similar device that is know to work well with passthrough. Or passthrough the individual drives instead of the controller.

 

[GenGT]

Thanks leesteken, these are bad news. I will try to virtually unplug and plug PCIe bus every time. It would be appreciated if someone could share some other workarounds, or hardware that don't have this problem. I need to passthrough the controller with 5 SATA ports because NAS software (xpenology in this case) need to control the health of the HDD (access to the smart data)