GPU and Bluetooth usb passthrough lost after some time

[debussy]

Hello,

I discovered Proxmox a couple of months ago and the experience has been really positive, it has improved the way I manage my self-hosted services that were previously running on a Raspberry Pi. However, I still have one problem and it is peripheral passthrough.

Problem: After 4-5 days my passthrough configuration is lost and my Jellyfin LXC container cannot not use GPU to decode video and my Ubuntu VM that is running Homebridge is not able to use Bluetooth USB device.

Workaround: Reboot proxmox server

Proxmox Version:

1. pve-manager/7.4-13/46c37d9c
2. Linux 6.2.11-2-pve

Hardware:

1. Intel NUC Intel(R) Core(TM) i7-1165G7
2. Bluetooth - SABRENT BT-UB40 (CSR8510A10)

Passthrough method:

1. GPU: Installed drivers in proxmox host and this is the config in LXC container

   lxc.cgroup2.devices.allow: alxc.cap.drop:
   lxc.cgroup2.devices.allow: c 226:0 rwm
   lxc.cgroup2.devices.allow: c 226:128 rwm
   lxc.cgroup2.devices.allow: c 29:0 rwm
   lxc.mount.entry: /dev/fb0 dev/fb0 none bind,optional,create=file
   lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir
   lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file
   lxc.cgroup2.devices.allow: c 10:200 rwm
   lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file

2. Bluetooth: Using VM hardware configuration UI I added the specific USB device

Relevant info
When running lsusb in proxmox host I can't see the Bluetooth USB device

root@proxmox:~# lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

neither in the LXC container

debussy@application:~$ lsusb
Bus 009 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 010 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 002: ID 0627:0001 Adomax Technology Co., Ltd QEMU USB Tablet
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 008 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

when running lspci in proxmox host I see the gpu

root@proxmox:~# lspci -nnk
00:00.0 Host bridge [0600]: Intel Corporation 11th Gen Core Processor Host Bridge/DRAM Registers [8086:9a14] (rev 01)
        DeviceName: Onboard - Other
        Subsystem: Intel Corporation 11th Gen Core Processor Host Bridge/DRAM Registers [8086:3004]
        Kernel modules: igen6_edac
00:02.0 VGA compatible controller [0300]: Intel Corporation TigerLake GT2 [Iris Xe Graphics] [8086:9a49] (rev 01)
        DeviceName: Onboard - Video
        Subsystem: Intel Corporation Iris Xe Graphics [8086:3004]
        Kernel driver in use: vfio-pci
        Kernel modules: i915
00:06.0 PCI bridge [0604]: Intel Corporation 11th Gen Core Processor PCIe Controller [8086:9a09] (rev 01)
        Kernel driver in use: pcieport
00:07.0 PCI bridge [0604]: Intel Corporation Tiger Lake-LP Thunderbolt 4 PCI Express Root Port #0 [8086:9a23] (rev 01)
        Kernel driver in use: pcieport
00:07.3 PCI bridge [0604]: Intel Corporation Tiger Lake-LP Thunderbolt 4 PCI Express Root Port #3 [8086:9a29] (rev 01)
        Kernel driver in use: pcieport
00:08.0 System peripheral [0880]: Intel Corporation Device [8086:9a11] (rev 01)
        DeviceName: Onboard - Other
        Subsystem: Intel Corporation Device [8086:3004]
00:0d.0 USB controller [0c03]: Intel Corporation Tiger Lake-LP Thunderbolt 4 USB Controller [8086:9a13] (rev 01)
        DeviceName: Onboard - Other
        Kernel driver in use: xhci_hcd
        Kernel modules: xhci_pci
00:0d.2 USB controller [0c03]: Intel Corporation Tiger Lake-LP Thunderbolt 4 NHI #0 [8086:9a1b] (rev 01)
        DeviceName: Onboard - Other
        Subsystem: Device [2222:1111]
        Kernel driver in use: thunderbolt
        Kernel modules: thunderbolt
00:0d.3 USB controller [0c03]: Intel Corporation Tiger Lake-LP Thunderbolt 4 NHI #1 [8086:9a1d] (rev 01)
        DeviceName: Onboard - Other
        Subsystem: Device [2222:1111]
        Kernel driver in use: thunderbolt
        Kernel modules: thunderbolt
00:14.0 USB controller [0c03]: Intel Corporation Tiger Lake-LP USB 3.2 Gen 2x1 xHCI Host Controller [8086:a0ed] (rev 20)
        DeviceName: Onboard - Other
        Subsystem: Intel Corporation Tiger Lake-LP USB 3.2 Gen 2x1 xHCI Host Controller [8086:3004]
        Kernel driver in use: vfio-pci
        Kernel modules: xhci_pci
00:14.2 RAM memory [0500]: Intel Corporation Tiger Lake-LP Shared SRAM [8086:a0ef] (rev 20)
        DeviceName: Onboard - Other
        Kernel driver in use: vfio-pci
00:14.3 Network controller [0280]: Intel Corporation Wi-Fi 6 AX201 [8086:a0f0] (rev 20)
        DeviceName: Onboard - Ethernet
        Subsystem: Intel Corporation Wi-Fi 6 AX201 [8086:0074]
        Kernel driver in use: vfio-pci
        Kernel modules: iwlwifi
00:15.0 Serial bus controller [0c80]: Intel Corporation Tiger Lake-LP Serial IO I2C Controller #0 [8086:a0e8] (rev 20)
        DeviceName: Onboard - Other
        Subsystem: Intel Corporation Tiger Lake-LP Serial IO I2C Controller [8086:3004]
        Kernel driver in use: intel-lpss
        Kernel modules: intel_lpss_pci
00:16.0 Communication controller [0780]: Intel Corporation Tiger Lake-LP Management Engine Interface [8086:a0e0] (rev 20)
        DeviceName: Onboard - Other
        Subsystem: Intel Corporation Tiger Lake-LP Management Engine Interface [8086:3004]
        Kernel driver in use: mei_me
        Kernel modules: mei_me
00:17.0 SATA controller [0106]: Intel Corporation Device [8086:a0d3] (rev 20)
        DeviceName: Onboard - SATA
        Subsystem: Intel Corporation Device [8086:3004]
        Kernel driver in use: ahci
        Kernel modules: ahci
00:1c.0 PCI bridge [0604]: Intel Corporation Device [8086:a0bc] (rev 20)
        Kernel driver in use: pcieport
00:1c.6 PCI bridge [0604]: Intel Corporation Device [8086:a0be] (rev 20)
        Kernel driver in use: pcieport
00:1f.0 ISA bridge [0601]: Intel Corporation Tiger Lake-LP LPC Controller [8086:a082] (rev 20)
        DeviceName: Onboard - Other
        Subsystem: Intel Corporation Tiger Lake-LP LPC Controller [8086:3004]
00:1f.3 Audio device [0403]: Intel Corporation Tiger Lake-LP Smart Sound Technology Audio Controller [8086:a0c8] (rev 20)
        DeviceName: Onboard - Sound
        Subsystem: Intel Corporation Tiger Lake-LP Smart Sound Technology Audio Controller [8086:3004]
        Kernel modules: snd_hda_intel, snd_sof_pci_intel_tgl
00:1f.4 SMBus [0c05]: Intel Corporation Tiger Lake-LP SMBus Controller [8086:a0a3] (rev 20)
        DeviceName: Onboard - Other
        Subsystem: Intel Corporation Tiger Lake-LP SMBus Controller [8086:3004]
        Kernel driver in use: i801_smbus
        Kernel modules: i2c_i801
00:1f.5 Serial bus controller [0c80]: Intel Corporation Tiger Lake-LP SPI Controller [8086:a0a4] (rev 20)
        DeviceName: Onboard - Other
        Subsystem: Intel Corporation Tiger Lake-LP SPI Controller [8086:3004]
        Kernel driver in use: intel-spi
        Kernel modules: spi_intel_pci
01:00.0 Non-Volatile memory controller [0108]: Toshiba Corporation Device [1179:0116]
        Subsystem: Toshiba Corporation Device [1179:0001]
        Kernel driver in use: nvme
        Kernel modules: nvme
58:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS525A PCI Express Card Reader [10ec:525a] (rev 01)
        Subsystem: Intel Corporation RTS525A PCI Express Card Reader [8086:3004]
        Kernel driver in use: rtsx_pci
        Kernel modules: rtsx_pci
59:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I225-V [8086:15f3] (rev 03)
        Subsystem: Intel Corporation Ethernet Controller I225-V [8086:3004]
        Kernel driver in use: igc
        Kernel modules: igc

Any hint or idea where I should look for trying to understand where is the error would be much appreciated.

Thanks.

 

[debussy]

After some effort I was able to discover the root cause, it was a weekly backup of a VM I used to test GPU passthrough. Here's a breakdown of the important entries:

1. The backup of a virtual machine (VM) with ID 102 (qemu) is starting. This is indicated by the log entry:
Jul 02 01:01:37 proxmox pvescheduler[3946831]: INFO: Starting Backup of VM 102 (qemu).

2. There are some USB-related events and disconnections happening, as shown by the following log entries:

- Jul 02 01:01:37 proxmox QEMU[10713]: kvm: libusb_release_interface: -4 [NO_DEVICE] - Jul 02 01:01:37 proxmox QEMU[10713]: kvm: libusb_release_interface: -4 [NO_DEVICE] - Jul 02 01:01:37 proxmox kernel: xhci_hcd 0000:00:14.0: remove, state 4 - Jul 02 01:01:37 proxmox kernel: usb usb4: USB disconnect, device number 1 - Jul 02 01:01:37 proxmox kernel: xhci_hcd 0000:00:14.0: USB bus 4 deregistered - Jul 02 01:01:37 proxmox kernel: xhci_hcd 0000:00:14.0: remove, state 1 - Jul 02 01:01:37 proxmox kernel: usb usb3: USB disconnect, device number 1 - Jul 02 01:01:37 proxmox kernel: usb 3-2: USB disconnect, device number 2 - Jul 02 01:01:37 proxmox kernel: usb 3-10: USB disconnect, device number 3 - Jul 02 01:01:37 proxmox kernel: xhci_hcd 0000:00:14.0: USB bus 3 deregistered



3. The system is managing the Bluetooth support, as indicated by the log entry:
Jul 02 01:01:37 proxmox systemd[1]: Stopped target bluetooth.target - Bluetooth Support.