[SOLVED] IOMMU on ProLiant DL380 Gen10

[Craize]

Does anyone else have the same problem on a ProLiant DL380 Gen10 server where the IOMMU is recognized as inactive? On my previous Gen8 server I never had a problem like that.

Used hardware:

CPU: 2x Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz
RAM: 535GB
GPU: NVIDIA Tesla P4

Proxmox VE Version + Kernel:

Proxmox Virtual Environment 8.4.16
Kernel Version: Linux 6.5.13-6-pve

/etc/kernel/cmdline

root=ZFS=rpool/ROOT/pve-1 boot=zfs
intel_iommu=on

/etc/default/grub

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"
GRUB_CMDLINE_LINUX=""

/etc/modules

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

Perhaps there is a setting in the BIOS that I am overlooking besides the virtualization options?

 

[leesteken]

/etc/kernel/cmdline

Only the first line is used and all other lines are ignored.
What is the output of cat /proc/cmdline?

Perhaps there is a setting in the BIOS that I am overlooking besides the virtualization options?

Maybe the VT-d setting is not enabled? You are also running an old(ish) PVE version and a old Linux kernel. From kernel version 6.8 intel_iommu=on is no longer needed and the module vfio_virqfd no longer exists.

 

[Craize]

Only the first line is used and all other lines are ignored.
What is the output of cat /proc/cmdline?

Maybe the VT-d setting is not enabled? You are also running an old(ish) PVE version and a old Linux kernel. From kernel version 6.8 intel_iommu=on is no longer needed and the module vfio_virqfd no longer exists.

Sorry for my late response!

cat /proc/cmdline

initrd=\EFI\proxmox\6.5.13-6-pve\initrd.img-6.5.13-6-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs

VT-d is enabled!

I use the old PVE/Kernel version cause I dont have a new driver for my GPU that works.

 

[leesteken]

Please use CODE-tags instead of quotes for readability and to prevent the text from disappearing on quoting.

cat /proc/cmdline

See, intel_iommu=on is missing. Add that in the right place (as mentioned before or stated in the manual).

 

[beisser]

intel_iommu=on or amd_iommu=on shouldnt be necessary anymore as iommu is on per default in 6.x kernels. you even posted that in some threads yourself

i would concentrate on the bios here. im reasonably certain that vt-d or some other virtualization-features are disabled there. i dont have a hp server so im not sure of the structure of the bios there, but it has to be there. the kernel doesnt disable iommu unless its forced off or the bios has it disabled.

 

[leesteken]

intel_iommu=on or amd_iommu=on shouldnt be necessary anymore as iommu is on per default in 6.x kernels. you even posted that in some threads yourself

amd_iommu=on has always been invalid and never required. intel_iommu=on is only no longer required since kernel version 6.8: https://pve.proxmox.com/pve-docs-8/pve-admin-guide.html#_configuration_14 (and the OP is still running the probably unsupported by now kernel version 6.5).

 

[beisser]

my bad. i apologize. then it is needed indeed.

also see https://support.hpe.com/hpesc/publi...6-0901-0A72-00000000072B.html&docLocale=en_US and verify that vt-d is indeed enabled in bios.

 

[Craize]

Please use CODE-tags instead of quotes for readability and to prevent the text from disappearing on quoting.

See, intel_iommu=on is missing. Add that in the right place (as mentioned before or stated in the manual).

Sorry, I'll use code tags in the future!
That was the solution. I just had to adjust the line and now everything works. Thanks!