iGPU passthrough

[ausghostdog]

What I would like:
Intel quick sync working on VM

What I have :
Virtual Environment 7.3-3
Intel W-1350
Already passed through another PCI-e card, and that is working within a VM
Proxmox is loading with grub https://pve.proxmox.com/wiki/Host_Bootloader


What I have done
1. DMAR: IOMMU enabled

2. grub has iommu=pt intel_iommu=on i915.enable_gvt=1

3.moduels has been set

4. I have updated the initramfs

​

running lspci -v, this appears to be the igpu

lspci -n -s 00:02.0



following https://blog.quindorian.org/2018/03/building-a-2u-amd-ryzen-server-proxmox-gpu-passthrough.html/
I made the driver set to vfio


The issue I have :
I built a test vm

attaching the pci device, I see nothing in drop down on mdev, I have seen other posts where the mdev type is populated, if I try to boot the VM with the pci device attached it boots but I can not ping it or access it in anyway, if I remove the device, it works as normal

What am I missing ?

 

[susannegraf]

Consider the following steps as a resolution pertaining to the Proxmox PCI Passthrough GPU issue:

1) Deselect the Primary GPU setting.

2) Reconfigure your GRUB settings as follows:


GRUB_CMDLINE_LINUX_DEFAULT="quiet pcie_acs_override=downstream,multifunction" GRUB_CMDLINE_LINUX="intel_iommu=on intel_iommu=igfx_off iommu=pt vfio_iommu_type1.allow_unsafe_interrupts=1 kvm.ignore_msrs=1" GRUB_TERMINAL=console


Please ensure that you subsequently update your initramfs with the command:


update-initramfs -u -k all


Also, make sure to update GRUB:


update-grub


Follow these steps with a system reboot.

Here's a note specific to Intel's iGPU:

By default, the gfx device is mapped as a regular device. If a gfx device is equipped with a dedicated DMAR unit, the DMAR unit is bypassed when not enabling DMAR with this particular option. In such a situation, the gfx device will resort to using the physical address for DMA.

Greetings
Susanne

 

[ausghostdog]

Makin g the change

GRUB_CMDLINE_LINUX_DEFAULT="quiet pcie_acs_override=downstream,multifunction"
GRUB_CMDLINE_LINUX="intel_iommu=on intel_iommu=igfx_off iommu=pt vfio_iommu_type1.allow_unsafe_interrupts=1 kvm.ignore_msrs=1"
GRUB_TERMINAL=console

stopped my vms from booting with can not prepare pci pass-throug, iommu not present

change back allowed me to start the vms again.

 

[leesteken]

GRUB_CMDLINE_LINUX="intel_iommu=on intel_iommu=igfx_off iommu=pt vfio_iommu_type1.allow_unsafe_interrupts=1 kvm.ignore_msrs=1"
stopped my vms from booting with can not prepare pci pass-throug, iommu not present

change back allowed me to start the vms again.

intel_iommu=igfx_off excludes the integrated graphics from IOMMU, which would explain what you are seeing.

EDIT: I guess I was wrong about this. Maybe intel_iommu=igfx_off overwrites intel_iommu=on and I mistook it for intel_iommu=on,igfx_off.

 

[ausghostdog]

intel_iommu=igfx_off excludes the integrated graphics from IOMMU, which would explain what you are seeing.

A different VM that is not using the igpu failed to start, I pass it a different pcie device

 

[ausghostdog]

Still working on this, appears I may have made a mistake originally, I have now been able to boot the other VM, but with changes I made when trying to get this to work, I have lost the device group for the cpu

so

gives me

where if I enable the code given I get iommu group as -

 

[ausghostdog]

following https://3os.org/infrastructure/proxmox/gpu-passthrough/igpu-passthrough-to-vm/

I thinking I have this w orking now

 

[distante]

following https://3os.org/infrastructure/proxmox/gpu-passthrough/igpu-passthrough-to-vm/

I thinking I have this w orking now

I do not quite understand this part:

You will loose the ability to use the onboard graphics card to access the Proxmox's console since Proxmox won't be able to use the Intel's gpu

That means I will not be able to directly connect to my machine (cable + keyboard) if something is wrong and I can not ssh to it?

 

[WillGFC]

I do not quite understand this part:



That means I will not be able to directly connect to my machine (cable + keyboard) if something is wrong and I can not ssh to it?

When you pass any device through to a Vm, you detach it from the host OS so in the case of a GPU it means that GPU cannot be used by the host so you wouldn't have a video output, but you can still SSH into the host.

Also, the GPU only gets detached once the boot process begins so if you boot into recovery mode you can remove the setting that detaches the GPU if you're experiencing issues.


How often do any of us need to connect a monitor to proxmox beyond the initial setup anyway?

 

[xhk416x]

I do not quite understand this part:



That means I will not be able to directly connect to my machine (cable + keyboard) if something is wrong and I can not ssh to it?

I'm a bit late on this one but here we go.

The reason the igpu would be unavailable to the proxmox host is because that guide is specifically disabling the host from using it so it'll be fully passed through to the VM. I didn't do this with whatever guide I followed before.
I have a VM that I pass through the igpu just for transcoding and my proxmox host still has a physical video console.

After examining the differences between my grub cmd line and the one from that guide it's pretty obvious what parts actually disable the card

# My grub line. Broken into newlines for easy reading
GRUB_CMDLINE_LINUX_DEFAULT="quiet
intel_iommu=on
i915.enable_gvt=1
iommu=pt
pcie_acs_override=downstream,multifunction
video=efifb:off
video=vesa:off
vfio_iommu_type1.allow_unsafe_interrupts=1
kvm.ignore_msrs=1"

# The guides grub line
GRUB_CMDLINE_LINUX_DEFAULT="quiet
intel_iommu=on
iommu=pt
pcie_acs_override=downstream,multifunction
initcall_blacklist=sysfb_init
video=simplefb:off
video=vesafb:off
video=efifb:off
video=vesa:off
disable_vga=1
vfio_iommu_type1.allow_unsafe_interrupts=1
kvm.ignore_msrs=1
modprobe.blacklist=radeon,nouveau,nvidia,nvidiafb,nvidia-gpu,snd_hda_intel,snd_hda_codec_hdmi,i915"

You can see `disable_vga=1` and the `modprobe.blacklist=######` with the list of gpu drivers. Those are obvious items that are disabling the igpu.

Now I'm not sure what the potential ramifications are of the host and a VM sharing the igpu like I am, but tbh for my little homelab, I like retaining the console access.

The guide I followed (I really wish I saved the link) also had the following in /etc/modules-load.d/igpu.conf

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
mdev
kvmgt

The main thing of note is the two last modules which are not in the guide you linked. Not sure if those are important cause I haven't looked deeper into it yet but that's another discrepancy.