Troubleshooting KVM Virtual Machine Startup Error with PCIe Passthrough on DL380 GEN9 (PVE 8)

[zhihuiyuze]

Hi there,

I'm creating a KVM virtual machine on my DL380 GEN9 (PVE 8) and attempting to pass the network card (eth0) through PCIE to the virtual machine in order to improve the network performance of my OPNsense virtual machine.

I followed the instructions on the Wiki to configure this and confirmed that VT-d is already enabled.

When I try to start the virtual machine, I encounter the following error:

()kvm: -device vfio-pci,host=0000:02:00.0,id=hostpci0,bus=pci.0,addr=0x10: vfio 0000:02:00.0: failed to setup container for group 53: Failed to set iommu for container: Operation not permittedTASK ERROR: start failed: QEMU exited with code 1

Here's my GRUB configuration file:

root@home-2:~# cat /etc/default/grub
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
# info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet"
GRUB_CMDLINE_LINUX="intel_iommu=on"

Here's my KVM virtual machine configuration in 1000.conf:

root@home-2:/etc/pve/local/qemu-server# cat 1000.conf
boot: order=scsi0;ide2;net0
cores: 16
cpu: host
hostpci0: 0000:02:00.0
ide2: local:iso/OPNsense-23.7-dvd-amd64.iso,media=cdrom,size=1594414K
memory: 2048
meta: creation-qemu=8.0.2,ctime=1696097073
name: OPNsense
net0: virtio=82:EB:64:2B:AA:07,bridge=vmbr0
numa: 0
ostype: l26
scsi0: local-lvm:vm-1000-disk-0,iothread=1,size=100G
scsihw: virtio-scsi-single
smbios1: uuid=269cfa12-b818-4d94-9582-2683065e87d9
sockets: 1
vmgenid: 6e58a4a6-0bae-4759-904a-de1876b6919c

I've already confirmed that I've rebooted after modifying GRUB and executed update-initramfs -u -k all.

Could someone plz help me?

 

[leesteken]

I'm creating a KVM virtual machine on my DL380 GEN9 (PVE 8)
...
Failed to set iommu for container: Operation not permittedTASK ERROR: start failed: QEMU exited with code 1

Please search the forum for RMRR as this appears to be a common thing with HP servers. Here is a thread that might help.

 

[zhihuiyuze]

I have updated the kernel accoding to the thread you provided for me.
And also added

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on,relax_rmrr iommu=pt intremap=no_x2apic_optout"

When I started the servere, their was an error.

[Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR 38d is 330)

and starting a vm with pcie passthrough will cause the system shutdown

Linux home-2 6.2.16-15-pve #1 SMP PREEMPT_DYNAMIC PMX 6.2.16-15 (2023-09-28T13:53Z) x86_64 GNU/Linux