Code:
pveversion: pve-manager/7.3-3/c3928077 (running kernel: 6.1.2-1-pve)
I am trying to get my onboard USB4 40Gbps ports operating at full capacity. I noted that when passing through the port to a VM, devices were reporting that they were connected on a USB 2.0 port. Upon investigating on the host, using lsusb I identified that the devices in question were connected under Bus 003, which is reporting as as a Linux Foundation 2.0 root hub:
Code:
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Code:
lsusb -t
/: Bus 12.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/0p, 5000M
/: Bus 11.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
/: Bus 10.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 10000M
/: Bus 09.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M
/: Bus 08.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 10000M
/: Bus 07.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M
/: Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/5p, 20000M
|__ Port 5: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
/: Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/12p, 480M
|__ Port 5: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 6: Dev 3, If 6, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 6: Dev 3, If 4, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 6: Dev 3, If 2, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 6: Dev 3, If 0, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 6: Dev 3, If 7, Class=Human Interface Device, Driver=usbhid, 480M
|__ Port 6: Dev 3, If 5, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 6: Dev 3, If 3, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 6: Dev 3, If 1, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 10: Dev 4, If 0, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 11: Dev 5, If 2, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 11: Dev 5, If 0, Class=Vendor Specific Class, Driver=, 12M
/: Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/5p, 20000M
/: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/12p, 480M
|__ Port 5: Dev 2, If 2, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 5: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 5: Dev 2, If 1, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 6: Dev 3, If 1, Class=Wireless, Driver=btusb, 12M
|__ Port 6: Dev 3, If 0, Class=Wireless, Driver=btusb, 12M
|__ Port 11: Dev 4, If 0, Class=Vendor Specific Class, Driver=, 12M
|__ Port 11: Dev 4, If 1, Class=Human Interface Device, Driver=usbhid, 12M
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 10000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M
Code:
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 9 Hub
bDeviceSubClass 0
bDeviceProtocol 1 Single TT
bMaxPacketSize0 64
idVendor 0x1d6b Linux Foundation
idProduct 0x0002 2.0 root hub
bcdDevice 6.01
iManufacturer 3 Linux 6.1.2-1-pve xhci-hcd
iProduct 2 xHCI Host Controller
iSerial 1 0000:67:00.0
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 0x0019
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0xe0
Self Powered
Remote Wakeup
MaxPower 0mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 9 Hub
bInterfaceSubClass 0
bInterfaceProtocol 0 Full speed (or root) hub
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0004 1x 4 bytes
bInterval 12
Hub Descriptor:
bLength 11
bDescriptorType 41
nNbrPorts 12
wHubCharacteristic 0x000a
No power switching (usb 1.0)
Per-port overcurrent protection
TT think time 8 FS bits
bPwrOn2PwrGood 10 * 2 milli seconds
bHubContrCurrent 0 milli Ampere
DeviceRemovable 0x00 0x00
PortPwrCtrlMask 0xff 0xff
Hub Port Status:
Port 1: 0000.0100 power
Port 2: 0000.0100 power
Port 3: 0000.0100 power
Port 4: 0000.0100 power
Port 5: 0000.0103 power enable connect
Port 6: 0000.0103 power enable connect
Port 7: 0000.0100 power
Port 8: 0000.0100 power
Port 9: 0000.0100 power
Port 10: 0000.0100 power
Port 11: 0000.0103 power enable connect
Port 12: 0000.0100 power
can't get device qualifier: Resource temporarily unavailable
can't get debug descriptor: Resource temporarily unavailable
Device Status: 0x0001
Self Powered
Code:
lspci | grep USB
0e:00.0 USB controller: Intel Corporation Thunderbolt 4 NHI [Maple Ridge 4C 2020]
3a:00.0 USB controller: Intel Corporation Thunderbolt 4 USB Controller [Maple Ridge 4C 2020]
67:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 43f7 (rev 01)
69:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 43f7 (rev 01)
6b:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Device 15b6
6b:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Device 15b7
6c:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 15b8
Code:
lspci | grep Thunderbolt
0c:00.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
0d:00.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
0d:01.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
0d:02.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
0d:03.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
0e:00.0 USB controller: Intel Corporation Thunderbolt 4 NHI [Maple Ridge 4C 2020]
3a:00.0 USB controller: Intel Corporation Thunderbolt 4 USB Controller [Maple Ridge 4C 2020]
I considered passing through the Thunderbolt controllers themselves, however they all reside in IOMMU group 16, along with several other devices that the host requires to function. I am trying to avoid using the insecure acs_override workaround if possible, so it seems passing through these controllers directly is out of the question.
Code:
for d in $(find /sys/kernel/iommu_groups/16 -type l | sort -n -k5 -t/); do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU Group %s ' "$n"; lspci -nns "${d##*/}"; done;
IOMMU Group 16 04:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 16 06:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f4] (rev 01)
IOMMU Group 16 07:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 16 07:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 16 07:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 16 07:03.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 16 07:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 16 07:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 16 07:0c.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 16 07:0d.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 16 08:00.0 Network controller [0280]: Intel Corporation Device [8086:2725] (rev 1a)
IOMMU Group 16 09:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I225-V [8086:15f3] (rev 03)
IOMMU Group 16 0a:00.0 Ethernet controller [0200]: Aquantia Corp. Device [1d6a:94c0] (rev 03)
IOMMU Group 16 0b:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02)
IOMMU Group 16 0c:00.0 PCI bridge [0604]: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] [8086:1136] (rev 02)
IOMMU Group 16 0d:00.0 PCI bridge [0604]: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] [8086:1136] (rev 02)
IOMMU Group 16 0d:01.0 PCI bridge [0604]: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] [8086:1136] (rev 02)
IOMMU Group 16 0d:02.0 PCI bridge [0604]: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] [8086:1136] (rev 02)
IOMMU Group 16 0d:03.0 PCI bridge [0604]: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] [8086:1136] (rev 02)
IOMMU Group 16 0e:00.0 USB controller [0c03]: Intel Corporation Thunderbolt 4 NHI [Maple Ridge 4C 2020] [8086:1137]
IOMMU Group 16 3a:00.0 USB controller [0c03]: Intel Corporation Thunderbolt 4 USB Controller [Maple Ridge 4C 2020] [8086:1138]
IOMMU Group 16 67:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f7] (rev 01)
IOMMU Group 16 68:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f6] (rev 01)
I assume this is somehow driver related, but searching around for USB4 or Thunderbolt drivers, I'm coming up blank. Any suggestions? If there is any other information I can provide that may help, please let me know and I'll do my best to acquire it.