problem IOMMU

R

roger

Member
Jan 11, 2012
11
0
21
Hello,
I have a DELL blade with M600 blades.
I am trying to validate IOMMU on this type of server. Virtualization is enabled of course however I added in / etc / grub
GRUB_CMDLINE_LINUX_DEFAULT = "quiet intel_iommu = onvfio_iommu_type1.allow_unsafe_interrupts = 1"

but it doesn't work
Do you have any idea to activate IOMMU to directly attack the hardware and increase the performance of my servers?

Thank you
in advance
Eric
 
Please try just GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on", without the spaces in strange places.
Then run update-grub. Then reboot and check using cat /proc/cmdline. (If you use ZFS and UEFI this will do nothing.)
You can check your IOMMU groups afterwards using for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done.
 
hello thank you for the answer.
I carried out the commands and here is the result.

when I go to the VM and want to add PC devices, I am always asked to activate IOMMU (see photo)



I don't understand because it finds the devices well but I can't validate anything.

Thanks a lot for the help


IOMMU group * 00:00.0 Host bridge [0600]: Intel Corporation 5000P Chipset Memory Controller Hub [8086:25d8] (rev 92)
00:02.0 PCI bridge [0604]: Intel Corporation 5000 Series Chipset PCI Express x4 Port 2 [8086:25e2] (rev 92)
00:03.0 PCI bridge [0604]: Intel Corporation 5000 Series Chipset PCI Express x4 Port 3 [8086:25e3] (rev 92)
00:04.0 PCI bridge [0604]: Intel Corporation 5000 Series Chipset PCI Express x4 Port 4 [8086:25e4] (rev 92)
00:05.0 PCI bridge [0604]: Intel Corporation 5000 Series Chipset PCI Express x4 Port 5 [8086:25e5] (rev 92)
00:06.0 PCI bridge [0604]: Intel Corporation 5000 Series Chipset PCI Express x8 Port 6-7 [8086:25f9] (rev 92)
00:07.0 PCI bridge [0604]: Intel Corporation 5000 Series Chipset PCI Express x4 Port 7 [8086:25e7] (rev 92)
00:08.0 System peripheral [0880]: Intel Corporation 5000 Series Chipset DMA Engi ne [8086:1a38] (rev 92)
00:10.0 Host bridge [0600]: Intel Corporation 5000 Series Chipset FSB Registers [8086:25f0] (rev 92)
00:10.1 Host bridge [0600]: Intel Corporation 5000 Series Chipset FSB Registers [8086:25f0] (rev 92)
00:10.2 Host bridge [0600]: Intel Corporation 5000 Series Chipset FSB Registers [8086:25f0] (rev 92)
00:11.0 Host bridge [0600]: Intel Corporation 5000 Series Chipset Reserved Regis ters [8086:25f1] (rev 92)
00:13.0 Host bridge [0600]: Intel Corporation 5000 Series Chipset Reserved Regis ters [8086:25f3] (rev 92)
00:15.0 Host bridge [0600]: Intel Corporation 5000 Series Chipset FBD Registers [8086:25f5] (rev 92)
00:16.0 Host bridge [0600]: Intel Corporation 5000 Series Chipset FBD Registers [8086:25f6] (rev 92)
00:1d.0 USB controller [0c03]: Intel Corporation 631xESB/632xESB/3100 Chipset UH CI USB Controller #1 [8086:2688] (rev 09)
00:1d.1 USB controller [0c03]: Intel Corporation 631xESB/632xESB/3100 Chipset UH CI USB Controller #2 [8086:2689] (rev 09)
00:1d.2 USB controller [0c03]: Intel Corporation 631xESB/632xESB/3100 Chipset UH CI USB Controller #3 [8086:268a] (rev 09)
00:1d.7 USB controller [0c03]: Intel Corporation 631xESB/632xESB/3100 Chipset EH CI USB2 Controller [8086:268c] (rev 09)
00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev d 9)
00:1f.0 ISA bridge [0601]: Intel Corporation 631xESB/632xESB/3100 Chipset LPC In terface Controller [8086:2670] (rev 09)
01:00.0 PCI bridge [0604]: Intel Corporation 6311ESB/6321ESB PCI Express Upstrea m Port [8086:3500] (rev 01)
01:00.3 PCI bridge [0604]: Intel Corporation 6311ESB/6321ESB PCI Express to PCI- X Bridge [8086:350c] (rev 01)
02:00.0 PCI bridge [0604]: Intel Corporation 6311ESB/6321ESB PCI Express Downstr eam Port E1 [8086:3510] (rev 01)
02:01.0 PCI bridge [0604]: Intel Corporation 6311ESB/6321ESB PCI Express Downstr eam Port E2 [8086:3514] (rev 01)
03:00.0 PCI bridge [0604]: Broadcom EPB PCI-Express to PCI-X Bridge [1166:0103] (rev c3)
04:00.0 Ethernet controller [0200]: Broadcom Limited NetXtreme II BCM5708S Gigab it Ethernet [14e4:16ac] (rev 12)
05:00.0 PCI bridge [0604]: Broadcom EPB PCI-Express to PCI-X Bridge [1166:0103] (rev c3)
06:00.0 Ethernet controller [0200]: Broadcom Limited NetXtreme II BCM5708S Gigab it Ethernet [14e4:16ac] (rev 12)
08:00.0 RAID bus controller [0104]: LSI Logic / Symbios Logic MegaRAID SAS 1078 [1000:0060] (rev 04)
09:00.0 PCI bridge [0604]: Broadcom EPB PCI-Express to PCI-X Bridge [1166:0103] (rev c3)
0a:00.0 Ethernet controller [0200]: Broadcom Limited NetXtreme II BCM5708S Gigab it Ethernet [14e4:16ac] (rev 12)
0b:00.0 PCI bridge [0604]: Broadcom EPB PCI-Express to PCI-X Bridge [1166:0103] (rev c3)
0c:00.0 Ethernet controller [0200]: Broadcom Limited NetXtreme II BCM5708S Gigab it Ethernet [14e4:16ac] (rev 12)
0f:04.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] ES1000 [1002:515e] (rev 02)
 

Attachments

  • iommu.JPG
    iommu.JPG
    29.4 KB · Views: 4
It looks like IOMMU is not enabled, otherwise it would not show everything in IOMMU group * (notice the * instead of multiple groups with numbers).
Either it is turned off in the BIOS/UEFI or maybe you are using systemd-boot instead of GRUB? Or maybe it is something else...
Please show the output of cat /proc/cmdline.
 
Last edited:
cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.78-2-pve root=/dev/mapper/pve-root ro intel_iommu=on


I do not use uefi for my VM win10 (see photo)
 

Attachments

  • iommu2.JPG
    iommu2.JPG
    55.2 KB · Views: 1
I made this command compared to the site of proxmox documentation and it seems to be validated

dmesg | grep -e DMAR -e IOMMU
[ 0.260076] DMAR: IOMMU enabled
 
I was not talking about VMs, but it is now clear that your host uses GRUB to boot from LVM/ext4.
The system's IOMMU is not (fully) enabled. Without multiple IOMMU groups, you cannot use PCI passthrough.
What is the output of lsmod | egrep 'kvm|vfio'? Maybe you need to load the various kvm and vfio modules explicitly in /etc/modules?
Please check the manual of your Dell machine to find out how to turn all virtualization options on. Intel VT-d need to be supported and enabled for PCI passthrough (not just VT-x).
 
it is a DELL blade M600 bios 2.4 server but I did not see a setting to enable Vt-d


root@pve12:~# lsmod | egrep 'kvm|vfio'
kvm_intel 253952 0
kvm 659456 1 kvm_intel
vfio_pci 53248 0
vfio_virqfd 16384 1 vfio_pci
irqbypass 16384 2 vfio_pci,kvm
vfio_iommu_type1 32768 0
vfio 32768 2 vfio_iommu_type1,vfio_pci
root@pve12:~#
 
It looks like you did everything right but the machine does not separate IOMMU groups, which means you cannot use PCI passthrough.
I have no experience with Dell blades and don't know how to help you any further, sorry.
 
can you post the complete output of 'dmesg' ?
 
Hello,
the file is very long, what exactly do you want? plus i guess it's on the proxmox?
Thank you
 
You must log in or register to reply here.
Top Bottom