not able to activate iommu on zfs with uefi

[adminkc]

Hi

having real troubles solving this.

Setup:

1. proxmox 6.2-11
2. Operating System: Debian GNU/Linux 10 (buster)
3. Kernel: Linux 5.4.60-1-pve
4. NVIDIA Corporation TU104GL [Tesla T4]
5. Boot in UEFI mode
6. ZFS Installation

Mainly we would like to add nvidia Tesla4 PCI Passthrough.


According to:

1. https://pve.proxmox.com/wiki/Pci_passthrough#Enable_the_IOMMU
2. https://forum.proxmox.com/threads/zfs-efi-boot-raid-1-and-intel_iommu-on-boot-parameter.57212/

I basically need to edit the /etc/kernel/cmdline and run pve-efiboot-tool refresh (=spread out to ESPs)

So I added quiet intel_iommu=on and check /proc/cmdline
I do not succeed adding anything in /proc/cmdline

any suggestions/ideas what could be wrong?

thanx, best regards,
KC-IT-Admin

 

[Stoiko Ivanov]

* is the system indeed booted with uefi? (check by running `mount |grep efi` on the PVE-Host)
* maybe just add the parameters to '/etc/default/grub' - and run `update-grub` - if they are present afterwards this would either mean that the system is either booted in legacy mode, or does not have ZFS on Root

* you can edit the kernel commandline directly in the boot-loader (both systemd-boot and grub), by hitting 'e' on the boot-screen - try addin the parameters there.

I hope this helps!

 

[adminkc]

Hi

thank you very much for your support

* is the system indeed booted with uefi? (check by running `mount |grep efi` on the PVE-Host)

yes
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)


* maybe just add the parameters to '/etc/default/grub' - and run `update-grub` - if they are present afterwards this would either mean that the system is either booted in legacy mode, or does not have ZFS on Root

had pretty much very similar idea, so I tried that too, but unfortunately it also does not change anything in /proc/cmdline

* you can edit the kernel commandline directly in the boot-loader (both systemd-boot and grub), by hitting 'e' on the boot-screen - try addin the parameters there.

but then it is a one-timer. Will try that tomorrow, but the question is how to make it last.

I hope this helps!

thanx ... any further ideas ... whatever it is that comes to your mind ... please it surely helps!

best regards,
KC-IT-Team

 

[vesalius]

What exactly do you have in the /etc/kernel/cmdline file? I did not use the quiet option. This is what I have.

nano /etc/kernel/cmdline

root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt

Did you reboot after running

pve-efiboot-tool refresh

Barring that I would delve into your MB bios setting and make sure that VT-D and such are correctly setup.

 

[adminkc]

thank you for the Ideas and input ...

What exactly do you have in the /etc/kernel/cmdline file? I did not use the quiet option. This is what I have.

nano /etc/kernel/cmdline

root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt

tried that too;
unfortunately still nothing in /proc/cmdline and no still no iommu support

Did you reboot after running

pve-efiboot-tool refresh

Barring that I would delve into your MB bios setting and make sure that VT-D and such are correctly setup.

yes, I ran pve-efiboot-tool refresh

 

[Stoiko Ivanov]

unfortunately still nothing in /proc/cmdline and no still no iommu support

* check what's in the configuration of the boot-manager when booting (i.e. hit 'e' when the boot-screen pops up)
* add the 'intel_iommu=on' there
* if this works i would check from which disk the system actually boots and whether it's uuid is in '/etc/kernel/pve-efiboot-uuids'

I hope this helps!

 

[trevmlt]

Hi

thank you very much for your support



thanx ... any further ideas ... whatever it is that comes to your mind ... please it surely helps!

best regards,
KC-IT-Team

I know this is an old thread, but I thought this might help someone else. I had the same issue after my system completely crashed and I brought it back so it could boot using UEFI....system would boot but no IOMMU.

I found the solution when I mounted /dev/sda2 and had a look at loader/loader.conf.

I had to recreate the /etc/kernel/cmdline, and when I did that I typed the word "options". When I looked at /mnt/tmp/loader/entries/proxmox-x.x.xx-x-pve.conf the options line had "options options root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_IOMMU=on...." and nothing showing up in /proc/cmdline after doing "proxmox-boot-tool refresh". I deleted the word "options" from the /etc/kernel/cmdline executed the refresh again and everything was working again. System crashed after upgrade of a ZFS install to proxmox 7 and rebooting it with the bios set to legacy and uefi on a Asrock EP2c602 motherboard

 

[Zakikun]

I know this is an old thread, but I thought this might help someone else. I had the same issue after my system completely crashed and I brought it back so it could boot using UEFI....system would boot but no IOMMU.

I found the solution when I mounted /dev/sda2 and had a look at loader/loader.conf.

I had to recreate the /etc/kernel/cmdline, and when I did that I typed the word "options". When I looked at /mnt/tmp/loader/entries/proxmox-x.x.xx-x-pve.conf the options line had "options options root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_IOMMU=on...." and nothing showing up in /proc/cmdline after doing "proxmox-boot-tool refresh". I deleted the word "options" from the /etc/kernel/cmdline executed the refresh again and everything was working again. System crashed after upgrade of a ZFS install to proxmox 7 and rebooting it with the bios set to legacy and uefi on a Asrock EP2c602 motherboard

That's right. There's no "options" within /etc/kernel/cmdline. After a fresh install with zfs+uefi, it should be

root=ZFS=rpool/ROOT/pve-1 boot=zfs

. Then just add

intel_iommu=on

And it should works fine after reboot