GPU passthrough no longer working after update. 7.2.11 + 5.19 kernel

[dust_bro]

I've had GPU passthorugh working on this system with PVE 7.2.3 + 5.13 Kernel. When I updated to 5.15 Kernel, I couldn't get passthrough working without a script posted in the forum.
I just updated to PVE 5.2.11 and GPU passthrough stopped working all together. When I try to start the VM, I get an error that says

cannot prepare PCI pass-through, IOMMU not present

and the output of dmesg | grep -e DMAR -e IOMMU -e AMD-Vi is

[    0.014582] ACPI: DMAR 0x000000007BB87000 000088 (v02 INTEL  EDK2     00000002      01000013)
[    0.014618] ACPI: Reserving DMAR table memory at [mem 0x7bb87000-0x7bb87087]
[    0.199209] DMAR: Host address width 39
[    0.199211] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.199216] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[    0.199220] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.199224] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.199227] DMAR: RMRR base: 0x0000007e000000 end: 0x0000007f7fffff
[    0.199230] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.199232] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.199234] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.200773] DMAR-IR: Enabled IRQ remapping in x2apic mode

I updated the Kernel to 5.19 to see if that made a differece, but no change.
Then I updated the BIOS of my MOBO. No change.
Virtualization and VT-D are enabled in the BIOS.

They system is an MSI Z590-A Pro and i9-10850K'

What do you recommend I do to troubleshoot?

 

[leesteken]

Check again that VT-d is enabled in the BIOS and check that cat /proc/cmdline contains intel_iommu=on.

 

[dust_bro]

Thanks for the reply!
VT-d is on.

cat /proc/cmdline says:

initrd=\EFI\proxmox\5.19.7-1-pve\initrd.img-5.19.7-1-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs

I did add it to /etc/default/grub though.

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on initcall_blacklist=sysfb_init"
GRUB_CMDLINE_LINUX=""

 

[leesteken]

initrd=\EFI\proxmox\5.19.7-1-pve\initrd.img-5.19.7-1-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs

intel_iommu is not enabled. It was enabled by default in an earlier kernel but that change was reverted later.

I did add it to /etc/default/grub though.

Your system probably does not use GRUB but systemd-boot instead.

 

[dust_bro]

intel_iommu is not enabled. It was enabled by default in an earlier kernel but that change was reverted later.

Your system probably does not use GRUB but systemd-boot instead.

Does systemd-boot need to be installed separately?
I don't seem to have a current .conf file in the /boot/loader/entries/ directory.

EDIT:
It's installed, but I don't have a loader folder within boot

systemd is already the newest version (247.3-7+deb11u1)

 

[leesteken]

Does systemd-boot need to be installed separately?
I don't seem to have a current .conf file in the /boot/loader/entries/ directory.

Have you done the check as described in the manual? (note the orange link in the text)
What is the output of efibootmgr -v?
What is the output of proxmox-boot-tool status?
If your system does not use GRUB to boot, then systemd-boot is already installed and working.

A little further in the manual it describes how to edit the kernel parameters in /etc/kernel/cmdline when not using GRUB.

 

[dust_bro]

Have you done the check as described in the manual? (note the orange link in the text)

You were right. Originally, the system was booting Grub and then started using systemd-boot after the update.

Systemd-boot

The kernel commandline needs to be placed as one line in /etc/kernel/cmdline. To apply your changes, run proxmox-boot-tool refresh, which sets it as the option line for all config files in loader/entries/proxmox-*.conf.

Adding intel_iommu=on to the commandline and running proxmox-boot-tool refresh got me back up and running.
Thanks for the help!