Mediated vs Passthrough of iGPU

S

spetrillo

Member
Feb 15, 2024
284
15
18
Hello all,

I have an Intel iGPU that I would like to use in a Plex VM. It looks like I have two options. Has anyone dabbled here and which seems to be the most performant?

Thanks,
Steve
 
Which Intel CPU do you have? You can generally pass it through to a VM but you will lose the host's physical console unless you split the iGPU.

To support split iGPU so that you can still access the physical host console while passing it through to the VM you will need either:
  • I915 VT-G if you have a 6th to 10th gen CPU
  • I915 SR-IOV if you have 12th gen CPU and above
If you have 11th gen CPU then you are out of luck as it doesn't support either.

I run a 10th gen CPU and use VT-G to pass the iGPU through to an Ubuntu VM which has Plex running in a Docker container. It supports HW transcoding.
 
  • Like
Reactions: Onoitsu2
Yes I have an 8th gen iGPU, so all good there. More than likely I will go the split route, as I do not want to lose access to the physical host.
 
I've used split GPU on 8th and 10th Gen with I915 VT-G and it's pretty straight forward.

This is a pretty good guide.

https://3os.org/infrastructure/proxmox/gpu-passthrough/igpu-split-passthrough/

However, I only needed to use a shortened GRUB command line to get it to work:

GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on iommu=pt i915.enable_gvt=1"

My motherboard already puts the iGPU in its own IOMMU group so I didn't need to worry about pcie_acs_override=downstream,multifunction.

The settings video=efifb:off video=vesa:off are generally redundant now, vfio_iommu_type1.allow_unsafe_interrupts=1 is only required if there are issues but can also create some, and kvm.ignore_msrs=1 is only necessary if your logs get spammed with error messages.

You will need to assign a serial console the the Plex VM as you'll find the noVNC console will stop producing output once the i915 driver initialises (the host console will still work though).
 
Last edited:
  • Like
Reactions: luda83 and Onoitsu2
spetrillo said:
I would assume you use Grub bc you are using LVM as your storage manager? You said shortened command line? What is that?
Click to expand...
It accidentally posted before I'd finished typing it, and have edited the post to give more detail on why I use the shortened command line.

I actually use ZFS so I'm command line, but just noted the GRUB command so that it was consistent with what was used in the tutorial. If using ZFS it becomes:

root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt i915.enable_gvt=1
 
  • Like
Reactions: Onoitsu2
I am using this in my cmdline: quiet intel_iommu=on i915.enable_gvt=1 iommu=pt pcie_acs_override=downstream,multifunction

I was told that I needed to use the pcie override, so everything was in its own IOMMU group. I do not know if this is correct or not.
 
spetrillo said:
I am using this in my cmdline: quiet intel_iommu=on i915.enable_gvt=1 iommu=pt pcie_acs_override=downstream,multifunction

I was told that I needed to use the pcie override, so everything was in its own IOMMU group. I do not know if this is correct or not.
Click to expand...
The pcie_acs_override option is essentially a kernel patch that makes it appear as if the devices are in their own separate IOMMU group, however they are still physically linked.

If the device that you're wanting to pass through (in this case the iGPU) is in its own IOMMU group to begin with, then there is no need to add the pcie_acs_override option as it is only introducing something that can cause potential additional instability.

For example using the command for g in $(find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V); do echo "IOMMU Group ${g##*/}:"; for d in $g/devices/*; do echo -e "\t$(lspci -nns ${d##*/})"; done; done; shows that my iGPU (UHD Graphics 630), dGPU (T1000 8GB), and PCIe4 NVMe drive are naturally in their own IOMMU groups, my network controller (ConnectX-4 Lx) is in a shared group with a PCI bridge.

Code: 
IOMMU Group 0:
        00:02.0 VGA compatible controller [0300]: Intel Corporation CometLake-S GT2 [UHD Graphics 630] [8086:9bc8] (rev 03)
IOMMU Group 1:
        00:00.0 Host bridge [0600]: Intel Corporation Comet Lake-S 6c Host Bridge/DRAM Controller [8086:9b53] (rev 03)
IOMMU Group 2:
        00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 03)
        01:00.0 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] [15b3:1015]
IOMMU Group 3:
        00:14.0 USB controller [0c03]: Intel Corporation Tiger Lake-H USB 3.2 Gen 2x1 xHCI Host Controller [8086:43ed] (rev 11)
        00:14.2 RAM memory [0500]: Intel Corporation Tiger Lake-H Shared SRAM [8086:43ef] (rev 11)
IOMMU Group 4:
        00:15.0 Serial bus controller [0c80]: Intel Corporation Tiger Lake-H Serial IO I2C Controller #0 [8086:43e8] (rev 11)
IOMMU Group 5:
        00:16.0 Communication controller [0780]: Intel Corporation Tiger Lake-H Management Engine Interface [8086:43e0] (rev 11)
IOMMU Group 6:
        00:17.0 SATA controller [0106]: Intel Corporation Device [8086:43d2] (rev 11)
IOMMU Group 7:
        00:1b.0 PCI bridge [0604]: Intel Corporation Device [8086:43c4] (rev 11)
IOMMU Group 8:
        00:1c.0 PCI bridge [0604]: Intel Corporation Tiger Lake-H PCI Express Root Port #5 [8086:43bc] (rev 11)
IOMMU Group 9:
        00:1d.0 PCI bridge [0604]: Intel Corporation Tiger Lake-H PCI Express Root Port #9 [8086:43b0] (rev 11)
IOMMU Group 10:
        00:1f.0 ISA bridge [0601]: Intel Corporation B560 LPC/eSPI Controller [8086:4387] (rev 11)
        00:1f.3 Audio device [0403]: Intel Corporation Device [8086:f0c8] (rev 11)
        00:1f.4 SMBus [0c05]: Intel Corporation Tiger Lake-H SMBus Controller [8086:43a3] (rev 11)
        00:1f.5 Serial bus controller [0c80]: Intel Corporation Tiger Lake-H SPI Controller [8086:43a4] (rev 11)
        00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (14) I219-V [8086:15fa] (rev 11)
IOMMU Group 11:
        02:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU117GL [T1000 8GB] [10de:1ff0] (rev a1)
        02:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10fa] (rev a1)
IOMMU Group 12:
        04:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation PS5021-E21 PCIe4 NVMe Controller (DRAM-less) [1987:5021] (rev 01)

This because my motherboard puts the primary 16x PCIe slot in a shared IOMMU group and the second 16x PCIe slot, iGPU and M.2 PCIe into their own IOMMU groups.

As I'm wanting to pass through the dGPU I've put it in the second 16x PCIe slot so that it gets its own IOMMU group without needing to use the pcie_acs_override option. Given I just use the dGPU for Ai detection in Synology Surveillance Station it doesn't need the full 16x PCIe 3.0 bandwidth of the primary slot.

Try it without the pcie_acs_override option and play around with any cards that you want to pass through in different slots to see if you can get them on their own and avoid having to use the option.
 
  • Like
Reactions: Onoitsu2 and spetrillo
Yes I just realized why I have that in there. I have a 4 port Intel adapter in the only PCI slot I have. If I do not use it all 4 ports go into the same IOMMU group. Now that would be ok if I was using all 4 ports to passthrough to one VM, but I split two ports to one VM and two ports to another VM. The override parameter puts each port into its own IOMMU group, so I can select the individual ports in the VM config.

I am glad I do not need all that other garbage...ughh...that just confused me completely. KISS principle please!
 
  • Like
Reactions: kesawi
Makes sense then. Definitely agree with the KISS principle, although I find that I have a tendency to overcomplicate things at times with my setup just because I can (probably a common trait amongst most who have a home lab
 
I do the same as you. If I can complicate it I will...whether I know it or not! lol

Got a funky error in my VMs log about the i915:

Nov 24 15:30:35 plex01 kernel: i915 0000:00:10.0: [drm] *ERROR* Failed to disable SAGV (-110)
Nov 24 15:30:35 plex01 kernel: ------------[ cut here ]------------
Nov 24 15:30:35 plex01 kernel: i915 0000:00:10.0: drm_WARN_ON(i915->display.sagv.status == I915_SAGV_UNKNOWN)
Nov 24 15:30:35 plex01 kernel: WARNING: CPU: 0 PID: 382 at drivers/gpu/drm/i915/display/skl_watermark.c:116 intel_sagv_init+0x270/0x320 [i915]
Nov 24 15:30:35 plex01 kernel: Modules linked in: i915(+) cfg80211 drm_buddy binfmt_misc drm_display_helper cec rc_core nls_iso8859_1 i2c_algo_bit video wmi vmgenid i2c_piix4 joy>
Nov 24 15:30:35 plex01 kernel: CPU: 0 PID: 382 Comm: (udev-worker) Not tainted 6.8.0-49-generic #49-Ubuntu
Nov 24 15:30:35 plex01 kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 4.2023.08-4 02/15/2024
Nov 24 15:30:35 plex01 kernel: RIP: 0010:intel_sagv_init+0x270/0x320 [i915]
Nov 24 15:30:35 plex01 kernel: Code: 4c 8b 67 50 4d 85 e4 75 03 4c 8b 27 e8 f9 a8 db e5 48 c7 c1 20 aa e3 c0 4c 89 e2 48 c7 c7 d6 ab e5 c0 48 89 c6 e8 e0 c4 32 e5 <0f> 0b e9 de f>

In reading on these types of errors it says to ignore this...but its in red and I do not like red error messages. Do you see it in your log?

Thanks,
Steve
 
Cool...

I am all setup. Plex sees the iGPU and it's ready to do. This is neat. I can now take the Plex server app off my NAS and let the NAS serve out storage in various flavors. I am upgrading my NAS, replacing my 3.5 inch drives with 2.5 inch SSDs.

Thanks for the assist!
 
  • Like
Reactions: kesawi
No worries, glad to see you were able to get it working :)

The iGPU does a pretty good job of HW Transcoding for Plex and is much more power efficient than using a separate dGPU.
 
Kesawi...

Found something rather interesting and not sure why. Do you have vfio_mdev in your modules config file? If not add it, run update-initramfs -u -k all and reboot your PVE. Then go back and add the default monitor back to your mediated vm. The console works for me, and the GPU is still mediated. Not sure why but it is.
 
spetrillo said:
Found something rather interesting and not sure why. Do you have vfio_mdev in your modules config file? If not add it, run update-initramfs -u -k all and reboot your PVE. Then go back and add the default monitor back to your mediated vm. The console works for me, and the GPU is still mediated. Not sure why but it is.
Click to expand...

I don't think the vifo_mdev module exists from the 5.15 linux kernel.

Code: 
modprobe vfio-mdev
modprobe: FATAL: Module vfio-mdev not found in directory /lib/modules/6.8.12-5-pve
 
You must log in or register to reply here.
Top Bottom