Virtual GPU being disabled right after VM starts

[unknown-user]

Hello community,
So, I have this scenario for 12th gen Intel CPU:
- proxmox-ve_7.4-1 installed and updated (no ent repos)
- used this tutorial https://www.deskpool.com/quick-startup/intel-sriov-vgpu/ to enable virtual GPUs (just 2 instead of 7 mentioned inside)
- all up to point 3 of this tutorial works fine, all is being installed
- skipped point 3 and just prepared standard Win 11 VM (no additional PCIe devices)
- Windows was installed, renamed, prepared for remote connection and shutdown
- NOW: I added this new virtual GPU made of Intel iGPU, enabled those options: all functions, PCI express and ROM bar
- machine was started and then magic starts

Machine log said this TASK ERROR: no pci device info for device '0000:00:02.1' and dmesg said that i had 2 vGPUs and now I don't.
root@prox-test:~# dmesg | grep i915 [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.1.15-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on i915.enable_guc=3 i915.max_vfs=2 [ 0.057307] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.1.15-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on i915.enable_guc=3 i915.max_vfs=2 [ 6.730278] i915 0000:00:02.0: Running in SR-IOV PF mode [ 6.731233] i915 0000:00:02.0: [drm] VT-d active for gfx access [ 6.731235] i915 0000:00:02.0: vgaarb: deactivate vga console [ 6.731270] i915 0000:00:02.0: [drm] Using Transparent Hugepages [ 6.731879] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem [ 6.732535] mei_hdcp 0000:00:16.0-b638ab7e-94e2-4ea2-a552-d1c54b627f04: bound 0000:00:02.0 (ops i915_hdcp_component_ops [i915]) [ 6.732850] i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/adls_dmc_ver2_01.bin (v2.1) [ 6.738540] i915 0000:00:02.0: [drm] GT0: GuC firmware i915/tgl_guc_70.bin version 70.5.1 [ 6.738543] i915 0000:00:02.0: [drm] GT0: HuC firmware i915/tgl_huc.bin version 7.9.3 [ 6.740679] i915 0000:00:02.0: [drm] GT0: HuC: authenticated for all workloads! [ 6.740991] i915 0000:00:02.0: [drm] GT0: GUC: submission enabled [ 6.740992] i915 0000:00:02.0: [drm] GT0: GUC: SLPC enabled [ 6.741399] i915 0000:00:02.0: [drm] GuC RC: enabled [ 6.741937] mei_pxp 0000:00:16.0-fbf6fcf1-96cf-4e2e-a6a6-1bab8cbe36b1: bound 0000:00:02.0 (ops i915_pxp_tee_component_ops [i915]) [ 6.742002] i915 0000:00:02.0: [drm] Protected Xe Path (PXP) protected content support initialized [ 6.827079] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 1 [ 6.828025] snd_hda_intel 0000:00:1f.3: bound 0000:00:02.0 (ops i915_audio_component_bind_ops [i915]) [ 6.828105] i915 0000:00:02.0: 2 VFs could be associated with this PF [ 6.828553] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes [ 6.828756] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes [ 7.887320] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=io+mem [ 7.887371] i915 0000:00:02.1: enabling device (0000 -> 0002) [ 7.887385] i915 0000:00:02.1: Running in SR-IOV VF mode [ 7.887645] i915 0000:00:02.1: [drm] GT0: GUC: interface version 0.1.0.0 [ 7.889158] i915 0000:00:02.1: [drm] VT-d active for gfx access [ 7.889170] i915 0000:00:02.1: [drm] Using Transparent Hugepages [ 7.889554] i915 0000:00:02.1: [drm] GT0: GUC: interface version 0.1.0.0 [ 7.889708] i915 0000:00:02.1: GuC firmware PRELOADED version 1.0 submission:SR-IOV VF [ 7.889709] i915 0000:00:02.1: HuC firmware PRELOADED [ 7.891444] i915 0000:00:02.1: [drm] Protected Xe Path (PXP) protected content support initialized [ 7.891447] i915 0000:00:02.1: [drm] PMU not supported for this GPU. [ 7.891484] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.1 on minor 2 [ 7.891760] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=io+mem [ 7.891763] i915 0000:00:02.1: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none [ 7.891794] i915 0000:00:02.2: enabling device (0000 -> 0002) [ 7.891804] i915 0000:00:02.2: Running in SR-IOV VF mode [ 7.892170] i915 0000:00:02.2: [drm] GT0: GUC: interface version 0.1.0.0 [ 7.893097] i915 0000:00:02.2: [drm] VT-d active for gfx access [ 7.893105] i915 0000:00:02.2: [drm] Using Transparent Hugepages [ 7.893459] i915 0000:00:02.2: [drm] GT0: GUC: interface version 0.1.0.0 [ 7.893602] i915 0000:00:02.2: GuC firmware PRELOADED version 1.0 submission:SR-IOV VF [ 7.893603] i915 0000:00:02.2: HuC firmware PRELOADED [ 7.895276] i915 0000:00:02.2: [drm] Protected Xe Path (PXP) protected content support initialized [ 7.895278] i915 0000:00:02.2: [drm] PMU not supported for this GPU. [ 7.895319] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.2 on minor 3 [B][ 7.895439] i915 0000:00:02.0: Enabled 2 VFs[/B]
--> virtual GPU added and VM starting
--> below actions after I pushed start
[ 901.252309] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=io+mem [ 901.252319] i915 0000:00:02.2: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none [ 901.384158] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=io+mem [B][ 902.562572] i915 0000:00:02.0: Disabled 2 VFs[/B]

I was also trying this tutorial for Proxmox 8 release but this could not create any vGPUs.

Any ideas what went wrong or where is this bug which prevent running VM?

Thanks

 

[rexnebula]

Did you ever find a solution for this? I'm on Proxmox 8, have working VFs but as soon as I start a VM and vfio-pci does it's thing it disables all the i915 VF's

[  111.530677] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=io+mem
[  111.530685] i915 0000:00:02.2: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.530690] i915 0000:00:02.3: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.530693] i915 0000:00:02.4: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.530697] i915 0000:00:02.5: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.530701] i915 0000:00:02.6: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.530705] i915 0000:00:02.7: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none
[  111.646643] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=io+mem
[  111.646650] i915 0000:00:02.3: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.646654] i915 0000:00:02.4: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.646658] i915 0000:00:02.5: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.646661] i915 0000:00:02.6: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.646665] i915 0000:00:02.7: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.790718] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=io+mem
[  111.790725] i915 0000:00:02.4: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.790729] i915 0000:00:02.5: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.790732] i915 0000:00:02.6: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.790736] i915 0000:00:02.7: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.902534] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=io+mem
[  111.902542] i915 0000:00:02.5: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.902546] i915 0000:00:02.6: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  111.902549] i915 0000:00:02.7: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  112.042685] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=io+mem
[  112.042692] i915 0000:00:02.6: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  112.042696] i915 0000:00:02.7: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  112.154664] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=io+mem
[  112.154672] i915 0000:00:02.7: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  112.294506] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=io+mem:owns=io+mem
[  113.478969] i915 0000:00:02.0: Disabled 7 VFs
[  113.710734] vfio-pci 0000:00:02.0: vgaarb: deactivate vga console
[  113.710738] vfio-pci 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem