GPU Passthrough Error: Failed to set iommu for container: Operation not permitted

ayyEve

New Member
Jun 26, 2020
9
0
1
24
Heya,

I have an hp dl380p g8 running proxmox 6.1-7
cpu: 2x xeon e5 2620
ram: 94GB ecc
drives:
os: sata ssd 512gb
data: 1.2tb sas drive

I'm trying to pass through a quadro 600 (also tried 950 but same results, everything below is relating to the quadro) but I keep getting the error in the title, no matter what I try.

the gpu and its audio device are in the same group but separate from everything else
Code:
IOMMU Group 23:
        0a:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF108GL [Quadro 600] [10de:0df8] (rev a1)
        0a:00.1 Audio device [0403]: NVIDIA Corporation GF108 High Definition Audio Controller [10de:0bea] (rev a1)

this is currently my grub line:
Code:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=10de:0df8,10de:0bea vfio-pci.ids=10de:0df8,10de:0bea"

i need insecure interrupts
Code:
root@proxmox:~# dmesg | grep 'remapping'
[    0.912539] DMAR-IR: Enabled IRQ remapping in xapic mode
[    0.912540] x2apic: IRQ remapping doesn't support X2APIC mode
and added the following according to the proxmox guide
Code:
root@proxmox:~# cat /etc/modprobe.d/iommu_unsafe_interrupts.conf
options vfio_iommu_type1 allow_unsafe_interrupts=1

my gpu audio device gives this issue but even the solutions for it have not worked:
Code:
[ 1176.469918] vfio-pci 0000:0a:00.1: DMAR: Device is ineligible for IOMMU domain attach due to platform RMRR requirement.  Contact your platform vendor.

i have tried using bios and uefi guests but that doesnt change anything
i even updated the bios to a recent version (2015 -> 2019) since there was a known issue

this doc shows the issue and the resolution, but the res for gpus (update the bios) did not work, i would try the one for nics but its confusing to me
https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=emr_na-c04781229+

guides i've followed:
https://pve.proxmox.com/wiki/Pci_passthrough
https://www.unrelatedshit.com/2016/02/19/enable-pci-passthrough-for-qemukvm/
https://www.jimmdenton.com/gpu-offloading-openstack/
https://forums.unraid.net/topic/75869-solved-issue-getting-gpu-passthrough-working-on-new-server/

ask me for any info and i will do what i can to get it.
I'm not great with Linux so there's a good chance i did something wrong

update: i have an old ati card that i was able to "passthrough" but i was unable to use it on kubuntu 20.04 (detected as pci device but not listed under graphics), there was no monitor output when set as primary gpu, and windows did not have drivers for it so i was unable to test there, i have a rx 470 on the way, should be here either friday or next monday

edit: fixed spacing
 
Last edited:

wolfgang

Proxmox Staff Member
Oct 1, 2014
6,496
494
103
Hi,

can you send the VM config to see your setting?

Code:
cat /etc/pve/qemu-server/<vmid>.com
 

ayyEve

New Member
Jun 26, 2020
9
0
1
24
just want to say i have fixed the issue (someone helped me follow that hp issue guide) but another issue has come up. should i make a new forum post or continue here?
 

tom

Proxmox Staff Member
Staff member
Aug 29, 2006
15,520
906
163
If you have a new issue, please do the following:
  1. Read documentation
  2. Search the forum for similar issues
  3. If you cannot find a solution, please open a new thread
 
  • Like
Reactions: ayyEve

Qazt23

New Member
Jul 10, 2020
3
0
1
31
Can i ask how you fixed the issue? I'm currently trying to run a vm on HP ML350 G6 and I am having the same issue when trying to passthrough a 1050ti. I am not gonna pretend I understood half the steps you took as I am a beginner to say the least. I have not been able to find any bios updates without subscribing to HP SUPPORT.

I have enabled IOMMU through Grub?

this is the error that was spit out at me

"kvm: -device vfio-pci,host=0000:0b:00.0,id=hostpci0.0,bus=pci.0,addr=0x10.0,rombar=0,x-vga=on,multifunction=on: vfio 0000:0b:00.0: failed to setup container for group 21: Failed to set iommu for container: Operation not permitted
start failed: QEMU exited with code 1"

EDIT

I also tried the VM config code "cat /ect/pve/qemu-server/<vmid>.com as posted by Wolfgang and recieved this reply

"-bash: vmid: No such file or directory"

So i missed a step somewhere...?

EDIT 2 so i tried "apt-get grub" it appears to have updated. still no luck in adding gpu to the vm
 
Last edited:

ayyEve

New Member
Jun 26, 2020
9
0
1
24
you need to replace <vmid> with the id of the vm you're trying pci passthrough on ;)

try following this guide step by step (you can ignore the virtio driver part, it didnt work for me)
https://www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/

if you're still having issues here how i fixed mine:
there is a free bios update from 2018-2019 to fix the intel security issues (this one should work for you https://support.hpe.com/hpsc/swd/public/detail?swItemId=MTX_3fb6b5c2b5184ef589331d2086), follow the installation instructions on that page
then i had to run opensuse 15 off a usb, install the hp-health package for suse 12 and the hp-scripting-tool for suse 15 and then i was able to follow the hp guide. to get the pcie slot number i just looked at my ilo:1594346626279.png
they show empty because i was using my machine as a test system before we purchased another machine :)

after i did all that, i tried passing through on a fresh windows 10 install and got it working with my nvidia 1070 and 950 (the amd r9 270 drivers would not install)

hopefully this helps!
 

Qazt23

New Member
Jul 10, 2020
3
0
1
31
@ayyEve Im just following your steps now, had to turn on the machine as we had a long power outtage after a big storm. Looks like proxmox has update and qemu. I figured id give it a try again, assigned the gpu and I now get a different error code...

"q35 machine model is not enabled at /usr/share/perl5/PVE/QemuServer.pm line 3585."

Im going to follow through with the "softpaq" update but have you seen this issue before?
 

ayyEve

New Member
Jun 26, 2020
9
0
1
24
from the proxmox ui:
click your vm -> hardware -> change "machine" from "i440fx" to "q35"

As per my previous post, you should follow that guide I sent step by step (preferably on a fresh proxmox install) and if you're still having issues let us know ;)
 

Qazt23

New Member
Jul 10, 2020
3
0
1
31
So update. I updated the firmware. The machine no longer turns on! This is becoming more fun at every step!
 

ayyEve

New Member
Jun 26, 2020
9
0
1
24
So update. I updated the firmware. The machine no longer turns on! This is becoming more fun at every step!
yikes, thats not good. im 80% sure the dl380p has a backup bios, not sure how to switch it though
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!