iGPU passthrough : PTE Write access is not set

[lxiosjao]

Hi,
I'm trying to passthrough the iGPU UHD graphic 630 to an Ubuntu VM.
Spec of the server : HP ELITEDESK 800 G4 (sff) using ZFS for the host (Systemd-boot)
The hardware seems to support IOMMU.
- CPU intel : i5-8500
- PVE : 7.4-3
- Kernel : 5.15.102

VM info :
- Bios : SeaBIOS
- Machine : q35

I followed this doc : https://pve.proxmox.com/pve-docs/chapter-qm.html#_general_requirements

1. Enabling IOMMU in the BIOS (vt-d) : ok
2. Enabling the IOMMU in the kernel command line + enabling the IOMMU passthrough mode (file : /etc/kernel/cmdline)

root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt

3. Apply the change for the kernel

proxmox-boot-tool refresh

4. Load the following modules (file : /etc/modules)

# Modules required for PCI passthrough
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

5. Refresh the initramfs :

update-initramfs -u -k all

5. Check if IOMMU is enabled with :

dmesg | grep -e DMAR -e IOMMU -e AMD-Vi

[    0.050647] DMAR: IOMMU enabled

6. Passthrough the iGPU to the VM


7. Restarting the VM : the VM is stuck, unable to login into the shell


Debug with :

dmesg -w

Any idea ?
Thanks

 

[lxiosjao]

UP

 

[celemine1gig]

Try to use the saved Legacy VBIOS file (option "romfile=") and use legacy boot on your VM. This seems to have the best bet to get this working. At least it does work for my own Win10 VM on CoffeeLake.

 

[lxiosjao]

Try to use the saved Legacy VBIOS file (option "romfile=") and use legacy boot on your VM. This seems to have the best bet to get this working. At least it does work for my own Win10 VM on CoffeeLake.

Thanks for your suggestion. Do you have documentation to do that ?

 

[celemine1gig]

See here for how to retrieve the vbios file:
https://pve.proxmox.com/wiki/PCI_Passthrough

Additional hint:
In order to save the legacy vbios, you will most likely need to make sure, that CSM is enabled in your system BIOS and that the "Video" Sub-Option for the CSM-ROM settings is actually set to "Legacy". If it is set to "UEFI" you cannot directly save the Legacy VBIOS.
After those steps, when you successfully saved the VBIOS file, you can most likely set the BIOS options back to the way they were.

 

[lxiosjao]

I read the doc from the link but I don't understand why I have to do that on this hardware. Can you explain more ?
PVE started with the legacy mode (not with UEFI) does it matter ?

 

[Clintos]

To those who end up here via google search after me.

In the UI go to the VM you're trying to pass the device to, click hardware, double click machine, turn on advanced, change vIOMMU from VirtIO to Intel (AMD Compatible), click ok, launch your machine.