Running: 5.3.18-2-pve #1 SMP PVE 5.3.18-2 on an AMD Ryzen 9 3900X
I picked up a PEXUS313AC2V card to to pass through to a Windows guest that I need to attach a bunch of USB devices to. Passthrough sort of worked, windows saw both controllers on the card but with the default windows driver the controller would fail with error 43 as soon as something was plugged in. Installing Startech's driver resulted in Windows BSODing about a minute after boot if the card was attached to the VM.
The PCIe card shows up as two PCI devices 10:00 and 11:00, each are in their own IOMMU group.
lspci -v: https://gist.github.com/edalquist/0f3ce158cbb8b8403cbf53b47efbf3b5#file-lspci
IOMMU Groups: https://gist.github.com/edalquist/0f3ce158cbb8b8403cbf53b47efbf3b5#file-iommu
Kernel Command Line:
I do successfully pass through a Radeon 5700XT to this same VM but for testing/debugging I removed the PCI mapping from the VM config.
I gave up on passthrough and figured I'd just see if I could use the extra ports from Linux. After remove the vfio-pci ids from /etc/modprobe.d/vfio.conf I'm seeing the following in dmesg related to the two PCI devices.
I'd love to get this card passed through to the Windows VM but at this point I'd settle for figuring out why Linux doesn't like it.
I picked up a PEXUS313AC2V card to to pass through to a Windows guest that I need to attach a bunch of USB devices to. Passthrough sort of worked, windows saw both controllers on the card but with the default windows driver the controller would fail with error 43 as soon as something was plugged in. Installing Startech's driver resulted in Windows BSODing about a minute after boot if the card was attached to the VM.
The PCIe card shows up as two PCI devices 10:00 and 11:00, each are in their own IOMMU group.
lspci -v: https://gist.github.com/edalquist/0f3ce158cbb8b8403cbf53b47efbf3b5#file-lspci
IOMMU Groups: https://gist.github.com/edalquist/0f3ce158cbb8b8403cbf53b47efbf3b5#file-iommu
Kernel Command Line:
Code:
root=ZFS=rpool/ROOT/pve-1 boot=zfs amd_iommu=on iommu=pt video=efifb:off rootdelay=10 acpi_enforce_resources=lax
I do successfully pass through a Radeon 5700XT to this same VM but for testing/debugging I removed the PCI mapping from the VM config.
I gave up on passthrough and figured I'd just see if I could use the extra ports from Linux. After remove the vfio-pci ids from /etc/modprobe.d/vfio.conf I'm seeing the following in dmesg related to the two PCI devices.
Code:
# dmesg | grep -e 10:00 -e 11:00 -e 1b21:1242
[ 0.818796] pci 0000:10:00.0: [1b21:1242] type 00 class 0x0c0330
[ 0.818831] pci 0000:10:00.0: reg 0x10: [mem 0xf7a00000-0xf7a07fff 64bit]
[ 0.818890] pci 0000:10:00.0: enabling Extended Tags
[ 0.818974] pci 0000:10:00.0: PME# supported from D3hot D3cold
[ 0.819124] pci 0000:11:00.0: [1b21:1242] type 00 class 0x0c0330
[ 0.819159] pci 0000:11:00.0: reg 0x10: [mem 0xf7900000-0xf7907fff 64bit]
[ 0.819219] pci 0000:11:00.0: enabling Extended Tags
[ 0.819302] pci 0000:11:00.0: PME# supported from D3hot D3cold
[ 1.215773] pci 0000:10:00.0: Adding to iommu group 37
[ 1.215795] pci 0000:10:00.0: Using iommu direct mapping
[ 1.215871] pci 0000:11:00.0: Adding to iommu group 38
[ 1.215893] pci 0000:11:00.0: Using iommu direct mapping
[ 1.272338] xhci_hcd 0000:10:00.0: xHCI Host Controller
[ 1.272341] xhci_hcd 0000:10:00.0: new USB bus registered, assigned bus number 6
[ 1.331206] xhci_hcd 0000:10:00.0: hcc params 0x0200eec1 hci version 0x110 quirks 0x0000000000000010
[ 1.331487] usb usb6: SerialNumber: 0000:10:00.0
[ 1.331597] xhci_hcd 0000:10:00.0: xHCI Host Controller
[ 1.331599] xhci_hcd 0000:10:00.0: new USB bus registered, assigned bus number 7
[ 1.331601] xhci_hcd 0000:10:00.0: Host supports USB 3.1 Enhanced SuperSpeed
[ 1.331682] usb usb7: SerialNumber: 0000:10:00.0
[ 1.331805] xhci_hcd 0000:11:00.0: xHCI Host Controller
[ 1.331807] xhci_hcd 0000:11:00.0: new USB bus registered, assigned bus number 8
[ 1.390668] xhci_hcd 0000:11:00.0: hcc params 0x0200eec1 hci version 0x110 quirks 0x0000000000000010
[ 1.390956] usb usb8: SerialNumber: 0000:11:00.0
[ 1.391063] xhci_hcd 0000:11:00.0: xHCI Host Controller
[ 1.391065] xhci_hcd 0000:11:00.0: new USB bus registered, assigned bus number 9
[ 1.391067] xhci_hcd 0000:11:00.0: Host supports USB 3.1 Enhanced SuperSpeed
[ 1.391150] usb usb9: SerialNumber: 0000:11:00.0
[ 11.936324] xhci_hcd 0000:11:00.0: Abort failed to stop command ring: -110
[ 11.936346] xhci_hcd 0000:11:00.0: xHCI host controller not responding, assume dead
[ 11.936351] xhci_hcd 0000:11:00.0: HC died; cleaning up
[ 11.936365] xhci_hcd 0000:11:00.0: Timeout while waiting for setup device command
I'd love to get this card passed through to the Windows VM but at this point I'd settle for figuring out why Linux doesn't like it.