IOMMU unsafe interrupts enabled, still error message

Feni

Active Member
Jun 22, 2017
35
17
28
36
Hi there,

I've used Proxmox for years on an old HP DL380 G6 which needs a patch to disable IOMMU RMRR checking and allow unsafe interrupts in order to passthrough GPU's to VM's. This combination has worked all that time with some rewrites of my patch whenever another kernel is used in Proxmox (https://forum.proxmox.com/threads/c...ntel-iommu-driver-to-remove-rmrr-check.36374/).

After updating to Proxmox 6.1.8 (Focal) however, I get the following output in dmesg:
Code:
[vfio_iommu_type1_attach_group: No interrupt remapping support.  Use the module param "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform

A quick check shows that my modprobe.d conf files are still there:
Code:
root@alpha:/etc/modprobe.d# ls
blacklist.conf  iommu_unsafe_interrupts.conf  kvm.conf  pve-blacklist.conf  vfio.conf  zfs.conf

And unsafe interrupts should be enabled:
Code:
options vfio_iommu_type1 allow_unsafe_interrupts=1

Any guru here that can show me what I'm missing? Or tell me if the latest kernel has a new way (or no way) of dealing with unsafe iommu interrupts? I'm reading something about lockdown in v5.4, could that affect me?
 
Last edited:

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
5,176
1,491
164
South Tyrol/Italy
shop.proxmox.com
After updating to Proxmox 6.1.8 (Focal) however, I get the following output in dmesg:

There's no version named focal, and strictly seen the most current pve-manager version (at time of writing) in Proxmox VE is 6.1-8 :)
I guess that with "focal" you mean that you installed the experimental pve-kernel-5.4 series? Or does this happen with the default 5.3 one: pveversion -v

And unsafe interrupts should be enabled:

You can check the current active parameter value as root by looking in sysfs:
cat /sys/module/vfio_iommu_type1/parameters/allow_unsafe_interrupts

I'm reading something about lockdown in v5.4, could that affect me?

No, that specific thing should not affect you - we do not enable kernel lock down.

It rather seems that the "vfio_iommu_type1" is not a module by default in 5.4 anymore, i.e., compare:

Code:
ls -l /lib/modules/5.3.18-3-pve/kernel/drivers/vfio/
# with
ls -l /lib/modules/5.4.24-1-pve/kernel/drivers/vfio/

It switched from "build as module" to "build always in kernel", so modprobe doesn't loads that module (and cannot apply params) as it is already loaded due to being compiled directly into the kernel.

You should be able to set the option via the kernel commandline ( https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#sysboot_edit_kernel_cmdline )

Adding something like vfio_iommu_type1.allow_unsafe_interrupts=1 should do the trick.
 
  • Like
Reactions: janssensm and Feni

Feni

Active Member
Jun 22, 2017
35
17
28
36
There's no version named focal, and strictly seen the most current pve-manager version (at time of writing) in Proxmox VE is 6.1-8 :)
I guess that with "focal" you mean that you installed the experimental pve-kernel-5.4 series? Or does this happen with the default 5.3 one: pveversion -v



You can check the current active parameter value as root by looking in sysfs:
cat /sys/module/vfio_iommu_type1/parameters/allow_unsafe_interrupts



No, that specific thing should not affect you - we do not enable kernel lock down.

It rather seems that the "vfio_iommu_type1" is not a module by default in 5.4 anymore, i.e., compare:

Code:
ls -l /lib/modules/5.3.18-3-pve/kernel/drivers/vfio/
# with
ls -l /lib/modules/5.4.24-1-pve/kernel/drivers/vfio/

It switched from "build as module" to "build always in kernel", so modprobe doesn't loads that module (and cannot apply params) as it is already loaded due to being compiled directly into the kernel.

You should be able to set the option via the kernel commandline ( https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#sysboot_edit_kernel_cmdline )

Adding something like vfio_iommu_type1.allow_unsafe_interrupts=1 should do the trick.

Thanks for the great support Thomas, editing the kernel commandline worked indeed. :) My Linux fu is not that strong, my major was business economics.

5.4 (Focal Fossa) was installed after cloning the master pve-kernel.git and compiling a patch to remove RMRR check (see pve-kernel appendix *-removermrr below). This change will make many PCIe passthrough tutorials obsolete, including these two:
https://pve.proxmox.com/wiki/Pci_passthrough
https://pve.proxmox.com/wiki/PCI(e)_Passthrough

pveversion -v
Code:
proxmox-ve: 6.1-2 (running kernel: 5.4.24-1-pve-removermrr)
pve-manager: 6.1-8 (running version: 6.1-8/806edfe1)
pve-kernel-helper: 6.1-7
pve-kernel-5.3: 6.1-5
pve-kernel-5.0: 6.0-11
pve-kernel-5.4.24-1-pve-removermrr: 5.4.24-1
pve-kernel-5.3.18-2-pve: 5.3.18-2
pve-kernel-5.3.13-1-pve: 5.3.13-1
pve-kernel-5.3.13-1-pve-removermrr: 5.3.13-1
pve-kernel-5.3.10-1-pve-removermrr: 5.3.10-1
pve-kernel-5.3.10-1-pve-removermrr-v2: 5.3.10-1
pve-kernel-5.0.21-5-pve: 5.0.21-10
pve-kernel-5.0.21-1-pve-removermrr: 5.0.21-2
pve-kernel-5.0.15-1-pve: 5.0.15-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.3-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.15-pve1
libpve-access-control: 6.0-6
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.0-17
libpve-guest-common-perl: 3.0-5
libpve-http-server-perl: 3.0-5
libpve-storage-perl: 6.1-5
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 3.2.1-1
lxcfs: 3.0.3-pve60
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.1-3
pve-cluster: 6.1-4
pve-container: 3.0-22
pve-docs: 6.1-6
pve-edk2-firmware: 2.20200229-1
pve-firewall: 4.0-10
pve-firmware: 3.0-6
pve-ha-manager: 3.0-9
pve-i18n: 2.0-4
pve-qemu-kvm: 4.1.1-4
pve-xtermjs: 4.3.0-1
qemu-server: 6.1-7
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.3-pve1
 

davidthejr

New Member
May 13, 2020
1
0
1
27
You should be able to set the option via the kernel commandline ( https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#sysboot_edit_kernel_cmdline )

Adding something like vfio_iommu_type1.allow_unsafe_interrupts=1 should do the trick.


Hello! I have just ran into an issue with iommu not working after running updates on my proxmox box. where did you add this line to allow unsafe interrupts again? is it on the same line as the "intel_iommu=on" option? on its own line?

Thanks!
 

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
5,176
1,491
164
South Tyrol/Italy
shop.proxmox.com
Hi,

It should be added all in one line to the kernel command line, see the linked documentation article at the bottom:
https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#sysboot_edit_kernel_cmdline

But actually we moved the vfio-pci from builtin to module again, together with the xhci and similar modules (as they were they reason vfio-pci got switched builtin in the first place, as they could already bind some devices this way before vfio-pci could.)
https://pve.proxmox.com/pipermail/pve-devel/2020-May/043538.html

So with the next kernel release this change should not be required anymore, but if you change it now it will still stay valid with the (new) old way too - hope I didn't made to point to complicated :)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!