Warning: UPDATING these packages broke my pci passthrough.

AWP Technologies · Sunday at 07:07

I installed updates

libpve-common-perl/stable 8.2.6 all [upgradable from: 8.2.5]
pve-firmware/stable,stable 3.14-1 all [upgradable from: 3.13-3]
qemu-server/stable 8.2.5 amd64 [upgradable from: 8.2.4]

After doing this i was getting a task error saying cannot bind <pci-device> to vfio

Because my router is a vm with a pass through network card it screwed me. I had to bring out backup router and down grade all of these packages and then pass through was immediately restored.

I just wanted to post this to warn everyone and also let the devs know there is an issue with these packages.

I am on kernel Linux 6.8.12-3-pve
pve manager version: pve-manager/8.2.8

AWP Technologies · Sunday at 07:43

I forgot to mention that i don't know exactly which package fixed my issue. I updated all three
libpve-common-perl to 8.2.6
pve-firmware to 3.14-1
qemu-server to 8.2.5

Immediately after reboot vm wouldn't start because of the pci bind issue.

I then went down the list 1 by 1 downgrading and restarting host. It didn't start working again until i downgraded all three.

dcsapak · Monday at 11:24

could you try downgrading the 'libpve-common-perl' package to see if that would fix it?

you can do that with

Code:

apt install libpve-common-perl=8.2.5

EDIT: sorry i missed that you already downgraded...
did you try just the libpve-common-perl too?

ozioh · Monday at 15:31

Thank you very much sharing your experiences. This is exactly what happened to me.
All my vm's with pcie passthrough went down after the update. Finally, I have a clue regarding to what to do.

ozioh · Monday at 15:35

AWP Technologies said:
I installed updates

libpve-common-perl/stable 8.2.6 all [upgradable from: 8.2.5]
pve-firmware/stable,stable 3.14-1 all [upgradable from: 3.13-3]
qemu-server/stable 8.2.5 amd64 [upgradable from: 8.2.4]

After doing this i was getting a task error saying cannot bind <pci-device> to vfio

Because my router is a vm with a pass through network card it screwed me. I had to bring out backup router and down grade all of these packages and then pass through was immediately restored.

I just wanted to post this to warn everyone and also let the devs know there is an issue with these packages.

I am on kernel Linux 6.8.12-3-pve
pve manager version: pve-manager/8.2.8

Could you please share your steps how you downgraded them? Thank you.

ozioh · Monday at 15:50

Downgrading libpve-common-perl gave me errors therefore, I downgraded it last after rebooting my server with the other two downgrades. At that time, my pcie passthrough still was not working. After finally downgrading libpve-common-perl, everthing works. Thank you for the original poster for the info.

prondtoo · Monday at 16:57

The same thing happened to me.
I downgraded libpve-common-perl to 8.2.5 and qemu-server to 8.2.4.
And now my OpenWRT virtual machine with PCI passthrough can be started again.

dcsapak · Tuesday at 10:22

ozioh said:
Thank you very much sharing your experiences. This is exactly what happened to me.
All my vm's with pcie passthrough went down after the update. Finally, I have a clue regarding to what to do.

prondtoo said:
The same thing happened to me.
I downgraded libpve-common-perl to 8.2.5 and qemu-server to 8.2.4.
And now my OpenWRT virtual machine with PCI passthrough can be started again.

could you maybe post the exact error you encountered when it failed? We did recently change some internals on how we detect errors, and it seems some things are not necessary, so i just want to make sure which parts we should take care of differently

dcsapak · Tuesday at 10:24

@AWP Technologies , could you maybe also post the vm config that lead to the error?

ozioh · Tuesday at 11:22

dcsapak said:
could you maybe post the exact error you encountered when it failed? We did recently change some internals on how we detect errors, and it seems some things are not necessary, so i just want to make sure which parts we should take care of differently

Hi,
I have multiple VMs. So, in everyone one of them, I got a similar error to this one "TASK ERROR: Cannot bind 0000:0a:00.0 to vfio" This error was passing the hard disk control to VM 101. When I removed the pci device to test if it is going to work, I was getting another error for another pci device.

This error was related to gpu passthrough..
From journalctl:
Nov 04 08:25:10 proxmox pvedaemon[9694]: start VM 100: UPID

roxmox:000025DE:0003635B:6728D946:qmstart:100:root@pam:
Nov 04 08:25:10 proxmox pvedaemon[2119]: <root@pam> starting task UPID

roxmox:000025DE:0003635B:6728D946:qmstart:100>
Nov 04 08:25:10 proxmox pvedaemon[9694]: Cannot bind 0000:0b:00.3 to vfio
Nov 04 08:25:10 proxmox pvedaemon[2119]: <root@pam> end task UPID

roxmox:000025DE:0003635B:6728D946:qmstart:100:root>

Thank you,

dcsapak · Tuesday at 12:03

mhmm can you post your vm config and `lspci -nnk` output ?

i find it weird that it cannot bind to vfio-pci, while that is AFAIK a requirement for it to work with passthrough in qemu...

ozioh · Tuesday at 12:26

Sure, here it is:
root@proxmox:~# cat /etc/pve/qemu-server/100.conf
agent: 1
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
balloon: 0
bios: ovmf
boot: order=hostpci0
cores: 24
cpu: host,hidden=1
efidisk0: local-lvm:vm-100-disk-0,efitype=4m,pre-enrolled-keys=1,size=4M
hostpci0: 0000:01:00,pcie=1
hostpci1: 0000:04:00,pcie=1
hostpci2: 0000:0b:00,pcie=1,x-vga=1
hostpci3: 0000:0c:00.0
hostpci4: 0000:08:00.3
hostpci5: 0000:0d:00.0
hostpci6: 0000:0f:00.4
machine: pc-q35-9.0
memory: 40000
meta: creation-qemu=7.1.0,ctime=1678275272
name: Windows
net0: virtio=9E:93:12:27:EF:C6,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: win11
scsihw: virtio-scsi-single
smbios1: uuid=a0dc6806-df35-433d-a554-cb154d02649f
sockets: 1
tpmstate0: local-lvm:vm-100-disk-1,size=4M,version=v2.0
usb0: host=291a:3369
vga: none
vmgenid: 1a3b1d67-328a-4cde-b681-40eea15939d9

root@proxmox:~# lspci -nnk
00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
Subsystem: ASUSTeK Computer Inc. Starship/Matisse Root Complex [1043:8808]
00:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
Subsystem: ASUSTeK Computer Inc. Starship/Matisse IOMMU [1043:8808]
00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
Subsystem: ASUSTeK Computer Inc. Starship/Matisse GPP Bridge [1043:8808]
Kernel driver in use: pcieport
00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
Subsystem: ASUSTeK Computer Inc. Starship/Matisse GPP Bridge [1043:8808]
Kernel driver in use: pcieport
00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
DeviceName: Onboard IGD
00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
Subsystem: ASUSTeK Computer Inc. Starship/Matisse GPP Bridge [1043:8808]
Kernel driver in use: pcieport
00:03.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
Subsystem: ASUSTeK Computer Inc. Starship/Matisse GPP Bridge [1043:8808]
Kernel driver in use: pcieport
00:03.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
Subsystem: ASUSTeK Computer Inc. Starship/Matisse GPP Bridge [1043:8808]
Kernel driver in use: pcieport
00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
Subsystem: ASUSTeK Computer Inc. Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1043:8808]
Kernel driver in use: pcieport
00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
Subsystem: ASUSTeK Computer Inc. Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1043:8808]
Kernel driver in use: pcieport
00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
Subsystem: ASUSTeK Computer Inc. FCH SMBus Controller [1043:87c0]
Kernel driver in use: piix4_smbus
Kernel modules: i2c_piix4, sp5100_tco
00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
Subsystem: ASUSTeK Computer Inc. FCH LPC Bridge [1043:87c0]
00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 0 [1022:1440]
00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 1 [1022:1441]
00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 2 [1022:1442]
00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 3 [1022:1443]
Kernel driver in use: k10temp
Kernel modules: k10temp
00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 4 [1022:1444]
00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 5 [1022:1445]
00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 6 [1022:1446]
00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 7 [1022:1447]
01:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation E18 PCIe4 NVMe Controller [1987:5018] (rev 01)
Subsystem: Phison Electronics Corporation E18 PCIe4 NVMe Controller [1987:5018]
Kernel driver in use: vfio-pci
Kernel modules: nvme
02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream [1022:57ad]
Kernel driver in use: pcieport
03:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3]
Subsystem: ASUSTeK Computer Inc. Matisse PCIe GPP Bridge [1043:8808]
Kernel driver in use: pcieport
03:03.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3]
Subsystem: ASUSTeK Computer Inc. Matisse PCIe GPP Bridge [1043:8808]
Kernel driver in use: pcieport
03:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3]
Subsystem: ASUSTeK Computer Inc. Matisse PCIe GPP Bridge [1043:8808]
Kernel driver in use: pcieport
03:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3]
Subsystem: ASUSTeK Computer Inc. Matisse PCIe GPP Bridge [1043:8808]
Kernel driver in use: pcieport
03:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4]
Subsystem: ASUSTeK Computer Inc. Matisse PCIe GPP Bridge [1043:8808]
Kernel driver in use: pcieport
03:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4]
Subsystem: ASUSTeK Computer Inc. Matisse PCIe GPP Bridge [1043:8808]
Kernel driver in use: pcieport
03:0a.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4]
Subsystem: ASUSTeK Computer Inc. Matisse PCIe GPP Bridge [1043:8808]
Kernel driver in use: pcieport
04:00.0 Non-Volatile memory controller [0108]: ADATA Technology Co., Ltd. XPG SX8200 Pro PCIe Gen3x4 M.2 2280 Solid State Drive [1cc1:8201] (rev 03)
Subsystem: ADATA Technology Co., Ltd. XPG SX8200 Pro PCIe Gen3x4 M.2 2280 Solid State Drive [1cc1:8201]
Kernel driver in use: vfio-pci
Kernel modules: nvme
05:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
Subsystem: ASUSTeK Computer Inc. RTL8125 2.5GbE Controller [1043:87d7]
Kernel driver in use: r8169
Kernel modules: r8169
06:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)
Subsystem: ASUSTeK Computer Inc. I211 Gigabit Network Connection [1043:85f0]
Kernel driver in use: igb
Kernel modules: igb
07:00.0 Network controller [0280]: Intel Corporation Wi-Fi 6 AX200 [8086:2723] (rev 1a)
Subsystem: Intel Corporation Wi-Fi 6 AX200NGW [8086:0084]
Kernel modules: iwlwifi
08:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
Subsystem: ASUSTeK Computer Inc. Starship/Matisse Reserved SPP [1043:8808]
Kernel driver in use: vfio-pci
08:00.1 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
Subsystem: ASUSTeK Computer Inc. Matisse USB 3.0 Host Controller [1043:8808]
Kernel driver in use: vfio-pci
Kernel modules: xhci_pci
08:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:148c]
Kernel driver in use: vfio-pci
Kernel modules: xhci_pci
09:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901]
Kernel driver in use: vfio-pci
Kernel modules: ahci
0a:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901]
Kernel driver in use: vfio-pci
Kernel modules: ahci
0b:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU104 [GeForce RTX 2070 SUPER] [10de:1e84] (rev a1)
Subsystem: ZOTAC International (MCO) Ltd. TU104 [GeForce RTX 2070 SUPER] [19da:4509]
Kernel driver in use: vfio-pci
Kernel modules: nvidiafb, nouveau
0b:00.1 Audio device [0403]: NVIDIA Corporation TU104 HD Audio Controller [10de:10f8] (rev a1)
Subsystem: ZOTAC International (MCO) Ltd. TU104 HD Audio Controller [19da:4509]
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
0b:00.2 USB controller [0c03]: NVIDIA Corporation TU104 USB 3.1 Host Controller [10de:1ad8] (rev a1)
Subsystem: ZOTAC International (MCO) Ltd. TU104 USB 3.1 Host Controller [19da:4509]
Kernel driver in use: vfio-pci
Kernel modules: xhci_pci
0b:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU104 USB Type-C UCSI Controller [10de:1ad9] (rev a1)
Subsystem: ZOTAC International (MCO) Ltd. TU104 USB Type-C UCSI Controller [19da:4509]
Kernel driver in use: vfio-pci
Kernel modules: i2c_nvidia_gpu
0c:00.0 Non-Volatile memory controller [0108]: MAXIO Technology (Hangzhou) Ltd. NVMe SSD Controller MAP1602 (DRAM-less) [1e4b:1602] (rev 01)
Subsystem: MAXIO Technology (Hangzhou) Ltd. NVMe SSD Controller MAP1602 (DRAM-less) [1e4b:1602]
Kernel driver in use: vfio-pci
Kernel modules: nvme
0d:00.0 Non-Volatile memory controller [0108]: MAXIO Technology (Hangzhou) Ltd. NVMe SSD Controller MAP1602 (DRAM-less) [1e4b:1602] (rev 01)
Subsystem: MAXIO Technology (Hangzhou) Ltd. NVMe SSD Controller MAP1602 (DRAM-less) [1e4b:1602]
Kernel driver in use: vfio-pci
Kernel modules: nvme
0e:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
Subsystem: ASUSTeK Computer Inc. Starship/Matisse PCIe Dummy Function [1043:8808]
0f:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
Subsystem: ASUSTeK Computer Inc. Starship/Matisse Reserved SPP [1043:8808]
Kernel driver in use: vfio-pci
0f:00.1 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP [1022:1486]
Subsystem: ASUSTeK Computer Inc. Starship/Matisse Cryptographic Coprocessor PSPCPP [1043:8808]
Kernel driver in use: ccp
Kernel modules: ccp
0f:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
Subsystem: ASUSTeK Computer Inc. Matisse USB 3.0 Host Controller [1043:87c0]
Kernel driver in use: xhci_hcd
Kernel modules: xhci_pci
0f:00.4 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller [1022:1487]
Subsystem: ASUSTeK Computer Inc. Starship/Matisse HD Audio Controller [1043:87c4]
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel

BertQuodge · Tuesday at 13:22

Hi

I don't know if my issue is different, but I now have an problem with USB pass-through, since updating to 6.8.12-3-pve yesterday. i didn't want to start a new thread with a related problem, so apologies if my issue is different fro the OP.

I pass through a USB hard disk to PVE containers and a ConBee III, which is a ZigBee tty serial device, to a Linux VM (6.1.0-26-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.112-1). Since the PVE upgrade, my USB disk device does appear in the container's lsusb, but I'm also seeing hundreds of thousands of errors on the ProxMox console, which relate to the USB attached disk USB id and are:

Code:

Nov 05 11:00:36 ProxMox1 kernel: usb 1-1.3: usbfs: process 1735 (CPU 1/KVM) did not claim interface 0 before use
Nov 05 11:00:36 ProxMox1 kernel: usb 1-1.3: usbfs: process 1735 (CPU 1/KVM) did not claim interface 0 before use
Nov 05 11:00:36 ProxMox1 kernel: usb 1-1.3: usbfs: process 1735 (CPU 1/KVM) did not claim interface 0 before use
Nov 05 11:00:36 ProxMox1 kernel: usb 1-1.3: usbfs: process 1735 (CPU 1/KVM) did not claim interface 0 before use
Nov 05 11:00:36 ProxMox1 kernel: usb 1-1.3: usbfs: process 1735 (CPU 1/KVM) did not claim interface 0 before use
Nov 05 11:00:36 ProxMox1 kernel: usb 1-1.3: usbfs: process 1735 (CPU 1/KVM) did not claim interface 0 before use
Nov 05 11:00:36 ProxMox1 kernel: usb 1-1.3: usbfs: process 1735 (CPU 1/KVM) did not claim interface 0 before use
Nov 05 11:00:36 ProxMox1 kernel: usb 1-1.3: usbfs: process 1735 (CPU 1/KVM) did not claim interface 0 before use
Nov 05 11:00:36 ProxMox1 kernel: usb 1-1.3: usbfs: process 1735 (CPU 1/KVM) did not claim interface 0 before use
Nov 05 11:00:36 ProxMox1 kernel: usb 1-1.3: usbfs: process 1735 (CPU 1/KVM) did not claim interface 0 before use

The tty device also does appear under lsusb on the Linux VM runnig under PVE:
root@HomeControl:~# lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 0403:6015 Future Technology Devices International, Ltd Bridge(I2C/SPI/UART/FIFO) (my tty device)
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 002: ID 0627:0001 Adomax Technology Co., Ltd QEMU Tablet
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

However, the device often drops out and vanishes from the lsusb list.

I see the following messages in the Linux VM's dmesg every few seconds. These errors occur every few seconds so I've truncated but I've added the pattern I see at the end of the line:

Code:

[3356.082353] ftdi_sio ttyUSB0: usb_serial_generic_write_bulk_callback - nonzero urb status: 71  [I](22 of these)[/I]
[3356.082353 xhci_hcd 0000:02:1b:0: ERROR_Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 4 [I](1 or 2 of these)[/I]

The linux VM is up to date, patch wise, and was obtained from the PVE.

The config for the VM can be seen here:

Code:

root@ProxMox1:~#  cat /etc/pve/qemu-server/105.conf
boot: order=scsi0;ide2;net0
cores: 2
cpu: x86-64-v2-AES
ide2: Internal1TB:iso/debian-12.7.0-amd64-netinst.iso,media=cdrom,size=631M
memory: 8192
meta: creation-qemu=9.0.2,ctime=1726326215
name: HomeControl
net0: virtio=BC:24:11:9D:AE:C4,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: l26
scsi0: local-lvm:vm-105-disk-0,iothread=1,size=64G
scsihw: virtio-scsi-single
smbios1: uuid=xxx
sockets: 1
startup: order=1
usb2: host=0403:6015
vmgenid: xxxx

lsusb on Proxmox can be seen here:

Code:

root@ProxMox1:~# lsusb -t
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
    |__ Port 2: Dev 3, If 0, Class=Mass Storage, Driver=uas, 5000M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 3: Dev 3, If 0, Class=Vendor Specific Class, Driver=usbfs, 12M
        |__ Port 4: Dev 4, If 0, Class=Hub, Driver=hub/4p, 480M
            |__ Port 4: Dev 7, If 0, Class=Vendor Specific Class, Driver=usbfs, 12M
            |__ Port 2: Dev 5, If 2, Class=Human Interface Device, Driver=usbhid, 480M
            |__ Port 2: Dev 5, If 0, Class=Human Interface Device, Driver=usbhid, 480M
            |__ Port 2: Dev 5, If 1, Class=Human Interface Device, Driver=usbhid, 480M

The ProxMox PCi devices:

Code:

root@ProxMox1:~# lspci -nnk
00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) Processor Root Complex [1022:1576]
    Subsystem: Hewlett-Packard Company Family 15h (Models 60h-6fh) Processor Root Complex [103c:8158]
00:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) I/O Memory Management Unit [1022:1577]
    Subsystem: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) I/O Memory Management Unit [1022:1577]
00:01.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Wani [Radeon R5/R6/R7 Graphics] [1002:9874] (rev 88)
    Subsystem: Hewlett-Packard Company Wani [Radeon R5/R6/R7 Graphics] [103c:8158]
    Kernel driver in use: amdgpu
    Kernel modules: amdgpu
00:01.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Kabini HDMI/DP Audio [1002:9840]
    Subsystem: Hewlett-Packard Company Kabini HDMI/DP Audio [103c:8158]
    Kernel driver in use: snd_hda_intel
    Kernel modules: snd_hda_intel
00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) Host Bridge [1022:157b]
    DeviceName:  Onboard IGD
00:02.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) Processor Root Port [1022:157c]
    Subsystem: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) Processor Root Port [1022:1234]
    Kernel driver in use: pcieport
00:02.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) Processor Root Port [1022:157c]
    Subsystem: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) Processor Root Port [1022:1234]
    Kernel driver in use: pcieport
00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) Host Bridge [1022:157b]
00:08.0 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Carrizo Platform Security Processor [1022:1578]
    Subsystem: Advanced Micro Devices, Inc. [AMD] Carrizo Platform Security Processor [1022:1578]
00:09.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Carrizo Audio Dummy Host Bridge [1022:157d]
00:09.2 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) Audio Controller [1022:157a]
    Subsystem: Hewlett-Packard Company Family 15h (Models 60h-6fh) Audio Controller [103c:8158]
    Kernel driver in use: snd_hda_intel
    Kernel modules: snd_hda_intel
00:10.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] FCH USB XHCI Controller [1022:7914] (rev 20)
    Subsystem: Hewlett-Packard Company FCH USB XHCI Controller [103c:8158]
    Kernel driver in use: xhci_hcd
    Kernel modules: xhci_pci
00:11.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 49)
    Subsystem: Hewlett-Packard Company FCH SATA Controller [AHCI mode] [103c:8158]
    Kernel driver in use: ahci
    Kernel modules: ahci
00:12.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] FCH USB EHCI Controller [1022:7908] (rev 49)
    Subsystem: Hewlett-Packard Company FCH USB EHCI Controller [103c:8158]
    Kernel driver in use: ehci-pci
    Kernel modules: ehci_pci
00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 4a)
    Subsystem: Hewlett-Packard Company FCH SMBus Controller [103c:8158]
    Kernel driver in use: piix4_smbus
    Kernel modules: i2c_piix4, sp5100_tco
00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 11)
    Subsystem: Hewlett-Packard Company FCH LPC Bridge [103c:8158]
00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) Processor Function 0 [1022:1570]
00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) Processor Function 1 [1022:1571]
00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) Processor Function 2 [1022:1572]
00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) Processor Function 3 [1022:1573]
    Kernel driver in use: k10temp
    Kernel modules: k10temp
00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) Processor Function 4 [1022:1574]
    Kernel driver in use: fam15h_power
    Kernel modules: fam15h_power
00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 60h-6fh) Processor Function 5 [1022:1575]
01:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)
    Subsystem: Hewlett-Packard Company RTL8111/8168/8211/8411 PCI Express Gigabit Ethernet Controller [103c:8158]
    Kernel driver in use: r8169
    Kernel modules: r8169
02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
    Subsystem: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:0123]
    Kernel driver in use: r8169
    Kernel modules: r8169

I've not tried to downgrade anything yet, as was mentioned in the posts above, to see if this helps.

Thanks

antplugger · Tuesday at 21:38

+1 for me. I upgraded on the weekend and can not passthrough motherboard USB controllers, GPU working though.
I have not tried downgeading yet

dcsapak · Wednesday at 14:59

@ozioh thanks for the output. This is super strange, since it seems that the device is already bound to the vfio-pci driver. could you confirm that this is the case also after the first reboot?
if not, is it bound to the vfio-driver after a failed start attempt?

dcsapak · Wednesday at 14:59

@BertQuodge this seems like a different issue, so please could you open a new thread for that? (so we can have the discussions organized) thanks!

dcsapak · Wednesday at 15:15

if somebody could try to apply the following patch: https://lore.proxmox.com/pve-devel/20241105092421.774448-2-d.csapak@proxmox.com/
it would be interesting which file_write has what exact error in the task log when starting & failing with 'cannot bind to vfio-pci'

BertQuodge · Wednesday at 15:40

dcsapak said:
@BertQuodge this seems like a different issue, so please could you open a new thread for that? (so we can have the discussions organized) thanks!

Understood, will do, thanks for your help!

ozioh · Wednesday at 20:25

dcsapak said:
@ozioh thanks for the output. This is super strange, since it seems that the device is already bound to the vfio-pci driver. could you confirm that this is the case also after the first reboot?
if not, is it bound to the vfio-driver after a failed start attempt?

Sorry about the confusion. the results of "lspci -nnk" was after I rolled back libpve-common-perl/stable 8.2.6 all [upgradable from: 8.2.5]. Please ignore it if you wanted to see the results after the upgrade. Thank you.

dcsapak · Thursday at 09:13

yes, i wanted the output with the broken state. Since i could not reproduce it yet here, it would be good if anybody affected could post the info i requested, so we can properly fix it here (instead of just restoring the old "wrong" code where it works)

Warning: UPDATING these packages broke my pci passthrough.

Member

Member

Proxmox Staff Member

Active Member

Active Member

Active Member

New Member

Proxmox Staff Member

Proxmox Staff Member

Active Member

Proxmox Staff Member

Active Member

New Member

Member

Proxmox Staff Member

Proxmox Staff Member

Proxmox Staff Member

New Member

Active Member

Proxmox Staff Member