Hi,
I have passed through my gpu successfully without issues (the whole card to 1 vm). Recently I discovered I can also cut up the gpu and passthrough the gpu to multiple VMs using vgpu. I got this working on proxmox 7.1. After passing through the vgpu I wanted to also passthrough my onboard audio but passing through additional hardware when the vgpu is connected to the VM crashes proxmox.
It doesnt matter what additional hardware I try to pass through, I tried a lan port and usb controller, proxmox dies after passing through to a vm with a vgpu. When I try to passthrough the same devices to a VM that does not have a vgpu attached it all works .. so I have no clue why.
I have attached the vgpu by editing the vm conf file adding:
args: -device 'vfio-pci,sysfsdev=/sys/bus/mdev/devices/79c49aaf-652b-4dac-a136-227a07a39538,display=off,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,x-pci-vendor-id=0x10de,x-pci-device-id=0x1e30,x-pci-sub-vendor-id=0x10de,x-pci-sub-device-id=0x12ba' -uuid 79c49aaf-652b-4dac-a136-227a07a39538
I read somewhere that a vm.conf may only have 1 args line, does adding hardware through the proxmox webgui add a second args line to the config ? Should I try to add the usb controller to the args line I use for the vgpu ?
Below are my iommu groups:
Im trying to passthrough the usb controller in group 35.
I found some details on the device in iommu group 35
Now im trying to make a string to add to the args line but im stuck at finding out what the device type should be ?
How can I find out what the device type should be and what bus I should put in ?
-device usb-host,bus=xhci.0,vendorid=0x1d6b,productid=0x0003
Thanks!
I have passed through my gpu successfully without issues (the whole card to 1 vm). Recently I discovered I can also cut up the gpu and passthrough the gpu to multiple VMs using vgpu. I got this working on proxmox 7.1. After passing through the vgpu I wanted to also passthrough my onboard audio but passing through additional hardware when the vgpu is connected to the VM crashes proxmox.
It doesnt matter what additional hardware I try to pass through, I tried a lan port and usb controller, proxmox dies after passing through to a vm with a vgpu. When I try to passthrough the same devices to a VM that does not have a vgpu attached it all works .. so I have no clue why.
I have attached the vgpu by editing the vm conf file adding:
args: -device 'vfio-pci,sysfsdev=/sys/bus/mdev/devices/79c49aaf-652b-4dac-a136-227a07a39538,display=off,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,x-pci-vendor-id=0x10de,x-pci-device-id=0x1e30,x-pci-sub-vendor-id=0x10de,x-pci-sub-device-id=0x12ba' -uuid 79c49aaf-652b-4dac-a136-227a07a39538
I read somewhere that a vm.conf may only have 1 args line, does adding hardware through the proxmox webgui add a second args line to the config ? Should I try to add the usb controller to the args line I use for the vgpu ?
Below are my iommu groups:
Im trying to passthrough the usb controller in group 35.
Code:
IOMMU Group 0 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 10 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 11 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
IOMMU Group 12 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
IOMMU Group 12 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU Group 13 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0 [1022:1440]
IOMMU Group 13 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1 [1022:1441]
IOMMU Group 13 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2 [1022:1442]
IOMMU Group 13 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3 [1022:1443]
IOMMU Group 13 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4 [1022:1444]
IOMMU Group 13 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5 [1022:1445]
IOMMU Group 13 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6 [1022:1446]
IOMMU Group 13 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7 [1022:1447]
IOMMU Group 14 01:00.0 Non-Volatile memory controller [0108]: Kingston Technology Company, Inc. Device [2646:5013] (rev 01)
IOMMU Group 15 02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream [1022:57ad]
IOMMU Group 16 03:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3]
IOMMU Group 17 03:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3]
IOMMU Group 18 03:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3]
IOMMU Group 19 03:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4]
IOMMU Group 1 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
IOMMU Group 20 03:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4]
IOMMU Group 21 03:0a.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4]
IOMMU Group 22 04:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 [144d:a808]
IOMMU Group 23 05:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
IOMMU Group 24 05:00.1 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
IOMMU Group 25 05:00.2 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
IOMMU Group 26 05:00.3 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
IOMMU Group 27 07:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)
IOMMU Group 28 08:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
IOMMU Group 28 08:00.1 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
IOMMU Group 28 08:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
IOMMU Group 29 09:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
IOMMU Group 2 00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
IOMMU Group 30 0a:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
IOMMU Group 31 0b:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU117 [GeForce GTX 1650] [10de:1f82] (rev a1)
IOMMU Group 31 0b:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10fa] (rev a1)
IOMMU Group 32 0c:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
IOMMU Group 33 0d:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
IOMMU Group 34 0d:00.1 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP [1022:1486]
IOMMU Group 35 0d:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
IOMMU Group 36 lspci: -s: Invalid slot number
IOMMU Group 37 lspci: -s: Invalid slot number
IOMMU Group 3 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 4 00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 5 00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
IOMMU Group 6 00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 7 00:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 8 00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 9 00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]I found some details on the device in iommu group 35
Code:
lsusb:
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Code:
lsusb -t
/: Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M
Code:
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 3.10
bDeviceClass 9 Hub
bDeviceSubClass 0
bDeviceProtocol 3
bMaxPacketSize0 9
idVendor 0x1d6b Linux Foundation
idProduct 0x0003 3.0 root hub
bcdDevice 5.15
iManufacturer 3 Linux 5.15.7-1-pve xhci-hcd
iProduct 2 xHCI Host Controller
iSerial 1 0000:0d:00.3
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 0x001f
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0xe0
Self Powered
Remote Wakeup
MaxPower 0mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 9 Hub
bInterfaceSubClass 0
bInterfaceProtocol 0 Full speed (or root) hub
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0004 1x 4 bytes
bInterval 12
bMaxBurst 0
Hub Descriptor:
bLength 12
bDescriptorType 42
nNbrPorts 4
wHubCharacteristic 0x000a
No power switching (usb 1.0)
Per-port overcurrent protection
bPwrOn2PwrGood 50 * 2 milli seconds
bHubContrCurrent 0 milli Ampere
bHubDecLat 0.0 micro seconds
wHubDelay 0 nano seconds
DeviceRemovable 0x00
Hub Port Status:
Port 1: 0000.02a0 lowspeed L1
Port 2: 0000.02a0 lowspeed L1
Port 3: 0000.02a0 lowspeed L1
Port 4: 0000.02a0 lowspeed L1
Binary Object Store Descriptor:
bLength 5
bDescriptorType 15
wTotalLength 0x0023
bNumDeviceCaps 2
SuperSpeed USB Device Capability:
bLength 10
bDescriptorType 16
bDevCapabilityType 3
bmAttributes 0x02
Latency Tolerance Messages (LTM) Supported
wSpeedsSupported 0x0008
Device can operate at SuperSpeed (5Gbps)
bFunctionalitySupport 1
Lowest fully-functional device speed is Full Speed (12Mbps)
bU1DevExitLat 0 micro seconds
bU2DevExitLat 0 micro seconds
SuperSpeedPlus USB Device Capability:
bLength 20
bDescriptorType 16
bDevCapabilityType 10
bmAttributes 0x00000001
Sublink Speed Attribute count 1
Sublink Speed ID count 0
wFunctionalitySupport 0x1104
bmSublinkSpeedAttr[0] 0x00050034
Speed Attribute ID: 4 5Gb/s Symmetric RX SuperSpeed
bmSublinkSpeedAttr[1] 0x000500b4
Speed Attribute ID: 4 5Gb/s Symmetric TX SuperSpeed
can't get debug descriptor: Resource temporarily unavailable
Device Status: 0x0001
Self PoweredNow im trying to make a string to add to the args line but im stuck at finding out what the device type should be ?
How can I find out what the device type should be and what bus I should put in ?
-device usb-host,bus=xhci.0,vendorid=0x1d6b,productid=0x0003
Thanks!
Last edited: