QEMU 5 and Legacy IGD passthrough

sshaikh

Member
Apr 23, 2017
69
17
13
6.2 may have broken VMs that use legacy IGD passthrough.

This is somewhat related to https://forum.proxmox.com/threads/igd-passthrough-setup-broke-with-proxmox-6-2.69670, except legacy IGD PT also relied on a quirk in qemu. This quirk used to be generally available, but since QEMU 5 has become optional at compile time (see this commit).

I'm therefore trying to build a new version of qemu-system-x86_64 that has the required config, but am getting a bit lost in the Proxmox source code. I've gotten as far as checking out https://git.proxmox.com/git/pve-qemu.git and updating all the submodules but am having trouble getting something to build.

I also need to know which config Proxmox is currently using, so any guidance would be appreciated.
 

Stefan_R

Proxmox Staff Member
Staff member
Jun 4, 2019
855
153
43
Vienna
This is somewhat related to https://forum.proxmox.com/threads/igd-passthrough-setup-broke-with-proxmox-6-2.69670, except legacy IGD PT also relied on a quirk in qemu. This quirk used to be generally available, but since QEMU 5 has become optional at compile time (see this commit).
Can't test this to say for certain right now, but from how I read the commit the config should be enabled by default if PCI support is enabled (which it definitely is). There isn't even an option in the configure file to set or unset this. This might be an upstream bug then, maybe building a version with the patch reverted might make sense for testing.

I'm therefore trying to build a new version of qemu-system-x86_64 that has the required config, but am getting a bit lost in the Proxmox source code. I've gotten as far as checking out https://git.proxmox.com/git/pve-qemu.git and updating all the submodules but am having trouble getting something to build.
Check out our developer docs for general build instructions. There shouldn't be anything QEMU specific, other than 'make deb' has to be run as root. For easiest workflow I'd recommend compiling on a Debian or even directly on a PVE installation.

I also need to know which config Proxmox is currently using, so any guidance would be appreciated.
See debian/rules in pve-qemu.
 

sshaikh

Member
Apr 23, 2017
69
17
13
Thanks for the reply. I've since made some progress but am now getting the following error trying to compile in a debian 10 container. I'll read the dev docs to see if there are any clues.

Code:
In file included from /root/pve-qemu/pve-qemu-kvm-5.0.0/include/qemu/timer.h:4,
                 from /root/pve-qemu/pve-qemu-kvm-5.0.0/include/qemu/timed-average.h:29,
                 from /root/pve-qemu/pve-qemu-kvm-5.0.0/include/block/accounting.h:28,
                 from /root/pve-qemu/pve-qemu-kvm-5.0.0/include/block/block_int.h:27,
                 from block/file-posix.c:30:
/usr/include/linux/swab.h: In function ‘__swab’:
/root/pve-qemu/pve-qemu-kvm-5.0.0/include/qemu/bitops.h:20:34: error: "sizeof" is not defined, evaluates to 0 [-Werror=undef]
 #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
                                  ^~~~~~
/root/pve-qemu/pve-qemu-kvm-5.0.0/include/qemu/bitops.h:20:41: error: missing binary operator before token "("
 #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
 

Stefan_R

Proxmox Staff Member
Staff member
Jun 4, 2019
855
153
43
Vienna
Ah, yeah, that's a known issue with the libc library actually. Try installing the package 'pve-kernel-libc-dev' from pve-no-subscription, that should fix that issue.
 

sshaikh

Member
Apr 23, 2017
69
17
13
Ah, thanks for the insight, that package did the trick.

Regarding the QEMU5 commit I understood the same, that it should be there with CONFIG_PC_PCI, so reverting the patch will be part of my investigation.

See debian/rules in pve-qemu.

I'm specifically looking for where for example CONFIG_PC_PCI might be defined. I'm still digging, but it would be a mak file somewhere.
 

Stefan_R

Proxmox Staff Member
Staff member
Jun 4, 2019
855
153
43
Vienna
I'm specifically looking for where for example CONFIG_PC_PCI might be defined. I'm still digging, but it would be a mak file somewhere.
default-configs/i386-softmmu.mak sets CONFIG_i440FX and CONFIG_Q35 which in turn sets CONFIG_PC_PCI via hw/i386/Kconfig
 

sshaikh

Member
Apr 23, 2017
69
17
13
So I've verified that the normal build pulls in the refactored IGD code, and that it still doesn't work.

Reverting the patch however does.

I'll follow up with upstream, but in the meantime would the pve patches be doing something? I do doubt it but I thought I'd ask.
 

Stefan_R

Proxmox Staff Member
Staff member
Jun 4, 2019
855
153
43
Vienna
So I've verified that the normal build pulls in the refactored IGD code, and that it still doesn't work.

Reverting the patch however does.
That's progress!

I'll follow up with upstream, but in the meantime would the pve patches be doing something? I do doubt it but I thought I'd ask.
If they push a follow-up fix let me know, I'm sure we could pick it up for pve-qemu if it's straightforward enough.

Our patches don't change anything regarding IGD AFAIK, but QEMU is a mighty beast so take that with a grain of salt. You could probably try this out by building only with the minimum required patches and testing with that (usually 0019-PVE-Add-dummy-id-command-line-parameter.patch alone is enough to make qemu-server happy).
 

sshaikh

Member
Apr 23, 2017
69
17
13
Instead of messing around with Proxmox patches I've reproduced the bug (and fix) in clean upstream sources. Filed a bug here:

https://bugs.launchpad.net/qemu/+bug/1882784

Thanks for the attention and help - I'm not sure if this should be marked solved as of now?

I'll be using a patched QEMU for this - should I just drop it in /usr/bin?
 

Stefan_R

Proxmox Staff Member
Staff member
Jun 4, 2019
855
153
43
Vienna
Instead of messing around with Proxmox patches I've reproduced the bug (and fix) in clean upstream sources. Filed a bug here:

https://bugs.launchpad.net/qemu/+bug/1882784
It's usually more effective to write directly to the qemu-devel mailing list, you could even submit the fix you mention in your report directly. If it gets accepted upstream we could definitely carry it, if that's really all there is to it.

Thanks for the attention and help - I'm not sure if this should be marked solved as of now?
However you want, I personally wouldn't since the matter is still open.

I'll be using a patched QEMU for this - should I just drop it in /usr/bin?
Either that or you can add it as a .patch file to debian/patches/pve/ and add it to debian/patches/series so you can build yourself a .deb package to install.
 

sshaikh

Member
Apr 23, 2017
69
17
13
My fix is simply defining CONFIG_VFIO_IGD which I'm not sure will wash with them ;). I'm not familiar enough with their process to see how to introduce a preprocessor variable from a kconfig variable.

But since Proxmox always has CONFIG_PCI I might make it a deb patch. If I do, and install, will it interfere with future Proxmox updates via apt?

EDIT: actually scrap that. The aarch64 target fails if I build it with IGD enabled, so it has to be driven by the kconfig. I'll figure something more general out eventually but for now I'll just build a patched qemu-system-x86_64.
 
Last edited:

Stefan_R

Proxmox Staff Member
Staff member
Jun 4, 2019
855
153
43
Vienna
But since Proxmox always has CONFIG_PCI I might make it a deb patch. If I do, and install, will it interfere with future Proxmox updates via apt?
it won't interfere, but updates on our side will override yours on upgrade. You can avoid it by setting "pve-qemu-kvm" on hold for example.
 

sshaikh

Member
Apr 23, 2017
69
17
13
Okay. Great - it being overridden is fine. In the meantime I figured out a patch that will work for all builds:

Diff:
diff --git a/hw/vfio/pci-quirks.c b/hw/vfio/pci-quirks.c
index 2d348f8237..a633df965e 100644
--- a/hw/vfio/pci-quirks.c
+++ b/hw/vfio/pci-quirks.c
@@ -25,6 +25,7 @@
 #include "hw/qdev-properties.h"
 #include "pci.h"
 #include "trace.h"
+#include "config-devices.h"

 /*
  * List of device ids/vendor ids for which to disable

The same fix was simultaneously offered by QEMU on the bug so it looks like it should be patched soon. I'm now interested in the deb build process so I'll see if I can get a deb patch working anyway.
 

tachang

New Member
Jun 29, 2020
8
0
1
39
What would be the best way to apply this patch? Wait for a Proxmox dot release or build QEMU from source?
 

Stefan_R

Proxmox Staff Member
Staff member
Jun 4, 2019
855
153
43
Vienna
The full fixes will be included in qemu-server 6.2-4 and pve-qemu-kvm 5.0.0-5, which should hit the pve-no-subscription repository soon. You can build the packages from source as well, if you don't want to wait (see https://git.proxmox.com).

With those packages installed, you can specify "legacy-igd=1" in your <vmid>.conf for the hostpci device to enable the new mode.
 
  • Like
Reactions: azeth

sshaikh

Member
Apr 23, 2017
69
17
13
If you already know how to build things then you'll have this fixed in less than 15 minutes.

If you don't then it's a fun (and safe enough) exercise to do that will take 2 hours or so.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!