gpu passthrough operation not permitted error

[SyntaxError210]

Hi all. Earlier today I posted a thread because I was having this issue with an HP Proliant dl360 g7 which turned out to be because of hp itself. So, I moved the gpu (radeon pro wx 5100) to a dell poweredge r710 and followed the guide again. I ended up getting the same error and after spending three days last time trying to fix it and ending up with the result that I couldn't because hp sucks, I would really like to not find out that dell has the same issue. I think it might be a problem with the gpu itself but I hope that's also wrong because its the only one we have. If anyone can help at all it would be greatly appreciated.
Grub settings:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"

dmesg | grep -e DMAR -e IOMMU output:

[    0.016479] ACPI: DMAR 0x000000009F3B3674 000240 (v01 DELL   PE_SC3   00000001 DELL 00000001)
[    0.352640] DMAR: IOMMU enabled
[    0.669178] DMAR-IR: This system BIOS has enabled interrupt remapping
[    1.800072] DMAR: Host address width 40
[    1.800074] DMAR: DRHD base: 0x000000af000000 flags: 0x0
[    1.800095] DMAR: dmar0: reg_base_addr af000000 ver 1:0 cap c90780106f0462 ecap f020fe
[    1.800096] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
[    1.800103] DMAR: dmar1: reg_base_addr fed90000 ver 1:0 cap c90780106f0462 ecap f020fe
[    1.800104] DMAR: RMRR base: 0x0000009f4c8000 end: 0x0000009f4dffff
[    1.800105] DMAR: RMRR base: 0x0000009f4b1000 end: 0x0000009f4bffff
[    1.800106] DMAR: RMRR base: 0x0000009f4a1000 end: 0x0000009f4a1fff
[    1.800107] DMAR: RMRR base: 0x0000009f4a3000 end: 0x0000009f4a3fff
[    1.800108] DMAR: RMRR base: 0x0000009f4a5000 end: 0x0000009f4a5fff
[    1.800109] DMAR: RMRR base: 0x0000009f4a7000 end: 0x0000009f4a7fff
[    1.800110] DMAR: RMRR base: 0x0000009f4a9000 end: 0x0000009f4a9fff
[    1.800110] DMAR: RMRR base: 0x0000009f4c0000 end: 0x0000009f4c0fff
[    1.800111] DMAR: RMRR base: 0x0000009f4c2000 end: 0x0000009f4c2fff
[    1.800112] DMAR: ATSR flags: 0x0
[    1.800437] DMAR: dmar0: Using Queued invalidation
[    1.800453] DMAR: dmar1: Using Queued invalidation
[    1.814212] DMAR: Intel(R) Virtualization Technology for Directed I/O
[    9.538835] AMD-Vi: AMD IOMMUv2 driver by Joerg Roedel <jroedel@suse.de>
[    9.538837] AMD-Vi: AMD IOMMUv2 functionality not available on this system
[  352.806986] vfio_iommu_type1_attach_group: No interrupt remapping support.  Use the module param "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform
[  593.049423] vfio_iommu_type1_attach_group: No interrupt remapping support.  Use the module param "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform

/etc/modules settings:

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

I followed the allow unsafe interrupts part of the guide with this command and it seemed to do what it was supposed to:

echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf

Finally, this is my 126.conf file (126 is the vmid):

bios: ovmf
bootdisk: ide0
cores: 3
efidisk0: HDDPool:126/vm-126-disk-1.qcow2,size=128K
hostpci0: 05:00,pcie=1,x-vga=1
ide0: HDDPool:126/vm-126-disk-0.qcow2,size=32G
ide2: SSDPool:iso/Windows.iso,media=cdrom
machine: q35
memory: 10272
name: windows-vm
net0: e1000=52:81:9E:A5:ED:89,bridge=vmbr1,firewall=1,tag=10
numa: 0
ostype: win10
scsihw: virtio-scsi-pci
smbios1: uuid=a931b443-8243-4b3b-b9ce-19ef8b8edcbb
sockets: 2
vmgenid: 527be331-79ff-4833-9d15-5d2cd804877e

This is the error that occurs when I attempt to start the vm:

kvm: -device vfio-pci,host=0000:05:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on: vfio 0000:05:00.0: failed to setup container for group 14: Failed to set iommu for container: Operation not permitted
TASK ERROR: start failed: QEMU exited with code 1

 

[leesteken]

Have you tried adding vfio_iommu_type1.allow_unsafe_interrupts=1 to the kernel parameters (after intel_iommu=on)? The error message in the logs suggests that this might help with this issue.

 

[SyntaxError210]

so I added that param, rebooted, and started the vm and it worked but there's no console connection.

 

[SyntaxError210]

I have no idea what to do. There's just no vnc connection.

 

[leesteken]

I have no idea what to do. There's just no vnc connection.

You added x-vga=1, which is necessary for NVidia card but also disables the virtual display, I think. If all went well, you should have output from the GPU and no VNC/Spice/virtual display. Therefore, this is to be expected, I think. However, your reaction seems to imply that you also have no output from the GPU? Maybe you can try with x-vga=0 to check if the GPU is seen by Windows (but installing the NVidia drivers might not work)?
EDIT: At least the passthrough appears to be working now for your system. I have no experience with Nvidia cards. Maybe search the forum or maybe someone else can help you here.

 

[SyntaxError210]

It works! Your right, the gpu is outputting the display. I had to bike a good bit to get to the physical servers so it took a bit to check but it is outputting through the gpu after 5 days of trying to get it to work. Thanks for all your help.