Hi All,
I’ve been reading through the forums about many users having issues with PCI-E passthrough after upgrading from Proxmox 5 to 6.
This issue has been caused by QEMU being upgraded to version 4 and the defaults for q35 having been changed.
After reading through the forum, it appears the main reason for upgrading to QEMU 4.0 was to fix other passthrough issues and get the benefit of a newer release which means the PCI-E root port hardware for q35 has also been changed.
The main change for q35 was to set split kernel IRQ support and enable Interrupt Remapping as default which has had major problems with vfio-pci INTx Support. The answer is to go back to version 3.1 support or re-enable the kernel only IRQ support and disable split IRQ support.
There are two workarounds which I’ve tested and both work
In either case, you will need to modify your machine type from the command line, like so:-
1) Specify the machine type as pc-q35-3.1 and therefore revert back to version 3.1 QEMU support for PCI-E passthrough.
qm set ID -machine pc-q35-3.1
2) Enable the kernel-based IRQ support.
qm set ID -args '-machine type=q35,kernel_irqchip=on'
Here are two interesting documents on q35 patch support for QEMU 4.0 and 4.1. For 4.1 a patch has been provided to revert back to kernel-based IRQ support due to the number of problems caused by the split IRQ support.
QEMU 4.0 q35 patch
https://patchwork.kernel.org/cover/10736915/
QEMU 4.1 v2 q35 patch
https://lists.sr.ht/~philmd/qemu/<155786484688.13873.6037015630912983760.stgit@gimli.home>
I personally think it is better to use QEMU 4.0 support and enable kernel-based IRQ support.
However, there is an issue with the latest Proxmox 6.0 release whereby the REGEX pattern needs updating for the machine parameter as it does not support comma-separated values like q35,kernel_irqchip=on.
So you get an annoying message all the time when trying to change something on a VM using the Proxmox
interface, that says Result Verification Failed (400) Machine: value does not match the regex pattern.
For your reference:-
The available parameters for q35 support in QEMU 4.0 are as follows: -
|--------------------------------------------------------+-------+----|
| params | split | IR |
|--------------------------------------------------------+-------+----|
| -M q35 | 0 | / |
| -M q35,accel=kvm | 1 | / |
| -M pc-q35-3.1,accel=kvm | 0 | / |
| -M q35,accel=kvm,kernel-irqchip=off | 0 | / |
| -M q35,accel=kvm,kernel-irqchip=on | 0 | / |
| -M q35 -device intel-iommu | 0 | 1 |
| -M q35,accel=kvm -device intel-iommu | 1 | 1 |
| -M q35,accel=kvm,kernel-irqchip=on -device intel-iommu | 0 | 0 |
|--------------------------------------------------------+-------+----|
As it appears to be such a big problem for so many users, how about patching the Proxmox interfaced so that these options are selectable from within the Proxmox Interface or at the very least fixing the REGEX to allow comma-separated values?
I’ve been reading through the forums about many users having issues with PCI-E passthrough after upgrading from Proxmox 5 to 6.
This issue has been caused by QEMU being upgraded to version 4 and the defaults for q35 having been changed.
After reading through the forum, it appears the main reason for upgrading to QEMU 4.0 was to fix other passthrough issues and get the benefit of a newer release which means the PCI-E root port hardware for q35 has also been changed.
The main change for q35 was to set split kernel IRQ support and enable Interrupt Remapping as default which has had major problems with vfio-pci INTx Support. The answer is to go back to version 3.1 support or re-enable the kernel only IRQ support and disable split IRQ support.
There are two workarounds which I’ve tested and both work
In either case, you will need to modify your machine type from the command line, like so:-
1) Specify the machine type as pc-q35-3.1 and therefore revert back to version 3.1 QEMU support for PCI-E passthrough.
qm set ID -machine pc-q35-3.1
2) Enable the kernel-based IRQ support.
qm set ID -args '-machine type=q35,kernel_irqchip=on'
Here are two interesting documents on q35 patch support for QEMU 4.0 and 4.1. For 4.1 a patch has been provided to revert back to kernel-based IRQ support due to the number of problems caused by the split IRQ support.
QEMU 4.0 q35 patch
https://patchwork.kernel.org/cover/10736915/
QEMU 4.1 v2 q35 patch
https://lists.sr.ht/~philmd/qemu/<155786484688.13873.6037015630912983760.stgit@gimli.home>
I personally think it is better to use QEMU 4.0 support and enable kernel-based IRQ support.
However, there is an issue with the latest Proxmox 6.0 release whereby the REGEX pattern needs updating for the machine parameter as it does not support comma-separated values like q35,kernel_irqchip=on.
So you get an annoying message all the time when trying to change something on a VM using the Proxmox
interface, that says Result Verification Failed (400) Machine: value does not match the regex pattern.
For your reference:-
The available parameters for q35 support in QEMU 4.0 are as follows: -
|--------------------------------------------------------+-------+----|
| params | split | IR |
|--------------------------------------------------------+-------+----|
| -M q35 | 0 | / |
| -M q35,accel=kvm | 1 | / |
| -M pc-q35-3.1,accel=kvm | 0 | / |
| -M q35,accel=kvm,kernel-irqchip=off | 0 | / |
| -M q35,accel=kvm,kernel-irqchip=on | 0 | / |
| -M q35 -device intel-iommu | 0 | 1 |
| -M q35,accel=kvm -device intel-iommu | 1 | 1 |
| -M q35,accel=kvm,kernel-irqchip=on -device intel-iommu | 0 | 0 |
|--------------------------------------------------------+-------+----|
As it appears to be such a big problem for so many users, how about patching the Proxmox interfaced so that these options are selectable from within the Proxmox Interface or at the very least fixing the REGEX to allow comma-separated values?
