Alder Lake GVT-d integrated graphics passthrough

jimjam

Member
Mar 2, 2022
34
7
8
37
Hello!
I am in the process of transferring a series of VMs from an ESXi build onto a new Alder Lake (i9-12900K) build with Proxmox.

In particular I am working on getting the integrated Intel graphics passed through to a Debian 11 VM for Plex transcoding.

I have been following GVT-g tutorials, but have come to the realization that 10th gen and up Intel CPUs do not support GVT-g anymore. They have moved on to GVT-d, the big difference being (if I understand correctly) that you can only pass the entire Integrated Graphics through to the VM.

I'm not entirely sure what to do for GVT-d support, so I have been following GVT-g tutorials and have sort of piece mealed this together. I am looking for guidance on if this is correct, and what I may be missing.

On the Proxmox host I have switched to the opt in kernel 5.15.19-2-pve.
In my Grub configuration file (/etc/default/grub) I have put the following settings into the GRUB_CMDLINE_LINUX_DEFUALT string:

915.force_probe=4680 intel_iommu=on i915.enable_gvt=1

The final string looks like this:

GRUB_CMDLINE_LINUX_DEFAULT="quiet i915.force_probe=4680 intel_iommu=on i915.enable_gvt=1"

From my understanding, this enables Linux Kernel support for Alder Lake and i915 GVT support.

I have also modified the /etc/modules file to load the following kernel modules:

Code:
# Modules required for PCI passthrough
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

# Modules required for Intel GVT
kvmgt
xengt
vfio-mdev

This is what dmesg says about i915 after boot:
dmesg after boot.png

I think this looks good, although I'm not really sure.

I proceed to attempt to pass i915 0000:00:02.0 PCI device through to my Plex server VM:
pci vm passthrough.png
There is no MDev Type because Alder Lake no longer has mediated devices, its all or nothing...???

I fire up my Debian 11 VM which has the 5.16.0-3-amd64 Linux kernel loaded. It appears as though 5.16 has the i915 probe enabled by default, so I do not need to modify the grub file to change how the Linux kernel is loaded.

dmesg on the VM after boot:
dmesg on VM after boot.png

Notice that the VBIOS tables were unable to be found (not sure what this means and can't seem to find much info about it. I have read something about dumping the VBIOS)

I can actually start a stream from Plex, and it appears to be hardware transcoding, but It quickly goes south and the VM ends up spewing this out:
crash after hardware transcoding.png

I'm not sure if this is happening because Alder Lake is relatively new, or if I have something setup incorrectly.

Any guidance would be appreciated.
 
Ok, I followed some of the links provided in my screen shots. I have found a firmware for Alder Lake i915 here: https://git.kernel.org/pub/scm/linu...x-firmware.git/tree/i915/adls_dmc_ver2_01.bin

I am going to try and get it to load properly to see if that keeps the system from crashing...

I found this debian image that claims to have other firmware binaries loaded into it. Im working on getting it loaded to see if it picks up the firmware.

https://cdimage.debian.org/cdimage/...cluding-firmware/weekly-builds/amd64/iso-dvd/

edit: That debian image did not load the firmware, I need to figure out how to manually do this step...
 
Last edited:
Simply doing:
apt install firmware-linux
made is so that the firmware loaded, but other issues arose:

failed to find VBIOS tables (VBT).png

In the interim I tested hardware decoding some videos on Plex and ran into the same problem as before...
 
  • Like
Reactions: majorgear
So I decided to reduce complexity and try and run Plex on the Proxmox host to see what would happen. Does hardware transcoding work well directly on the hardware? Short answer is no, it doesn't.

Essentially the same result, a reboot of the server is needed. I think I'm going to have to wait until Alder Lake support isn't so green.
 
gvtd is quite different to gvtg
If you are passing the whole igpu as in gvtd, you need to blacklist the drivers so the host does not use it , you should not be loading the drivers on the host, if the host grabs the igpu it will not be available to the guest vm.

remove the i915 grub parameters ( you might need to add video=efifb:eek:ff)

check the the guide here , https://pve.proxmox.com/wiki/Pci_passthrough , in the gpu passthrough section, the drivers i blacklisted are :-

blacklist snd_hda_intel
blacklist snd_hda_codec_hdmi
blacklist i915

If it’s setup correctly in order to pass the whole gpu you should see anything Relating to i915 in dmesg on the host as it should not be loading.
 
Last edited:
  • Like
Reactions: Jez2022
gvtd is quite different to gvtg
If you are passing the whole igpu as in gvtd, you need to blacklist the drivers so the host does not use it , you should not be loading the drivers on the host, if the host grabs the igpu it will not be available to the guest vm.

remove the i915 grub parameters ( you might need to add video=efifb:eek:ff)

check the the guide here , https://pve.proxmox.com/wiki/Pci_passthrough , in the gpu passthrough section, the drivers i blacklisted are :-

blacklist snd_hda_intel
blacklist snd_hda_codec_hdmi
blacklist i915

If it’s setup correctly in order to pass the whole gpu you should see anything Relating to i915 in dmesg on the host as it should not be loading.
i tried this, although i have it passthrough correctly to ubuntu, and ubuntu recognized it and installed MESA intel driver, but the desktop environment is running at 2~3fps....
https://forum.proxmox.com/threads/12th-gen-alderlake-gpu-passthrough-fail-slow-2-3-fps.105484/
 
Ok, I have been following through the PCI passthrough documentation getting more and more familiar with it. I have a new VM with OVMF (UEFI) Bios and a machine type of q35. I removed the grub commands from the host, except for the iommu=on, and my host has no i915 messages in dmesg anymore. I also set BAR resizing in my BIOS to disabled as directed to by the PCI passthrough page.

My VM has Debian 11 with the 5.16.0-3-amd64 and I get this error in dmesg:

1646754877162.png
 
look like you are making progress .
if you do ls -l /dev/dri , do you see card0 and renderD128 in the directory . (This is what is required for Plex HW transcoding)
Are you running Plex on the guest or in docker on the guest ?
If if docker you will need to pass /dev/dri , i.e
Code:
devices:
      - /dev/dri:/dev/dri
I also had to ensure the Plex docker had correct permissions to access this .
I did chmod 666 /dev/dri/*
 
I am running plex directly on the guest, no docker.

Here is the output from the ls -l /dev/dri command:
1646764815937.png
 
Do you have 2 GPU's or a vGPU also assigned ?
not sure why yours is showing Card and renderD129 also .
Have you tried running plex to check if it uses the gpu for HW transcode ?
 
I did have a vGPU assigned, I set it to none, and now there is only card0 and renderD128.

I also installed plex and added a library with some videos, started a stream and forced a transcode. It was hardware transcoding, but using the CPU it seems:
1646790632423.png
1646790657480.png
 
Ok, so something is wrong. My root partition on Proxmox was at 100% because my user.log reached a whopping 90gb. The log file was filled with these messages:

Code:
Mar  9 19:29:48 hardingconcepts QEMU[5482]: kvm: vfio_region_write(0000:00:02.0:region2+0x11da788, 0x0,4) failed: Device or resource busy
Mar  9 19:29:48 hardingconcepts QEMU[5482]: kvm: vfio_region_write(0000:00:02.0:region2+0x11da78c, 0x0,4) failed: Device or resource busy
Mar  9 19:29:48 hardingconcepts QEMU[5482]: kvm: vfio_region_write(0000:00:02.0:region2+0x11da790, 0x0,4) failed: Device or resource busy
Mar  9 19:29:48 hardingconcepts QEMU[5482]: kvm: vfio_region_write(0000:00:02.0:region2+0x11da794, 0x0,4) failed: Device or resource busy
Mar  9 19:29:48 hardingconcepts QEMU[5482]: kvm: vfio_region_write(0000:00:02.0:region2+0x11da798, 0x0,4) failed: Device or resource busy
Mar  9 19:29:48 hardingconcepts QEMU[5482]: kvm: vfio_region_write(0000:00:02.0:region2+0x11da79c, 0x0,4) failed: Device or resource busy
Mar  9 19:29:48 hardingconcepts QEMU[5482]: kvm: vfio_region_write(0000:00:02.0:region2+0x11da7a0, 0x0,4) failed: Device or resource busy
Mar  9 19:29:48 hardingconcepts QEMU[5482]: kvm: vfio_region_write(0000:00:02.0:region2+0x11da7a4, 0x0,4) failed: Device or resource busy
Mar  9 19:29:48 hardingconcepts QEMU[5482]: kvm: vfio_region_write(0000:00:02.0:region2+0x11da7a8, 0x0,4) failed: Device or resource busy
Mar  9 19:29:48 hardingconcepts QEMU[5482]: kvm: vfio_region_write(0000:00:02.0:region2+0x11da7ac, 0x0,4) failed: Device or resource busy
Mar  9 19:29:48 hardingconcepts QEMU[5482]: kvm: vfio_region_write(0000:00:02.0:region2+0x11da7b0, 0x0,4) failed: Device or resource busy
Mar  9 19:29:48 hardingconcepts QEMU[5482]: kvm: vfio_region_write(0000:00:02.0:region2+0x11da7b4, 0x0,4) failed: Device or resource busy
Mar  9 19:29:48 hardingconcepts QEMU[5482]: kvm: vfio_region_write(0000:00:02.0:region2+0x11da7b8, 0x0,4) failed: Device or resource busy
Mar  9 19:29:48 hardingconcepts QEMU[5482]: kvm: vfio_region_write(0000:00:02.0:region2+0x11da7bc, 0x0,4) failed: Device or resource busy

I did a BIOS update hoping that it would help, but it doesn't seem to have changed anything for me. I am still reading over the PCI and PCIE passthrough pages from Proxmox hoping something jumps off the page...
 
10th gen and earlier11th gen12th gen
Mediated passthrough using Windows hostyes using GVT-gyes with SR-IOV??
Windows drivers available
??
Full passthrough using Windows hostyes using GVT-dyes??same as 11th gen?
Mediated passthrough using Linux hostyes using GVT-gyes with SR-IOV?? but no drivers yetsame as 11th gen?
Full passthrough using Linux hostyes using GVT-dyes??same as 11th gen?

Trying to put things together... does that match your understanding?

Some other links for reference:
https://www.reddit.com/r/VFIO/comments/s726zt/tiger_lake_iris_xe_single_gpu_passthrough/
https://www.mail-archive.com/vfio-users@redhat.com/msg04030.html
 
Last edited:
My understanding is that GVT-d is all or nothing. There is no mediated passthrough with GVT-d. This is kinda reflected in your chart with the 10th gen cpus.

I have also been led to believe that both GVT-d and SR-IOV were to be available on Alder Lake, but there are no SR-IOV drivers available for Linux as of right now. So I have only been searching for GVT-d related information.
 
I'm also only interested in full passthrough/GVT-d. If GVT-d is still available then it should still work, but if that also needs SR-IOV then we're screwed for now.
 
I have not heard of or seen anyone but @scoobdriver be successful at this with Alder Lake. I’ve put about 16-18 hours in trying new configurations, different Linux kernels, different Linux versions, different bios settings, and different pro mix settings all with varying degrees of failure.
 
No I am not sure. I made an assumption that he did because of the title of this thread and my description. Shame on me :(
 
  • Like
Reactions: ________

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!