[SOLVED] Is PCIe Passthrough supposed to work with Proxmox 7?

Jan 27, 2018
45
8
13
41
I just tried to update a standalone (non-cluster) Proxmox 6.4 with 2 GPUs (1080 and 2080 SUPER) which are both passed through to different VMs. Maybe worth to mention I also pass through some USB controllers directly to those VMs. The mainboard in use is a X570 AORUS Pro. I blacklist GPU drivers so usually after the bootloader there is no more graphic output on the host.

However, after the update to Proxmox 7 the host showed a `EFI stub: loaded initrd from command line option` message after the bootloader. When I tried to start VMs with PCIe passthrough this went away and the screen went black again (so something happened) but it just stayed black. So I didn't even get the logo of the Proxmox boot menu inside the VM.

This was the case for all VMs with PCIe passthrough, Linux as well as Windows guests and for both GPUs, no matter how often I tried. The syslog and dmesg didn't show any to me obvious error message and looked normal. All VMs without any PCIe passthrough started and worked normally.

I haven't looked into it very long and decided to just rollback to 6.4 very soon (luckily I was prepared and now everything works again). Now I'm just wondering is this some kind of known problem or something specific to my system/unknown so far?

Thanks for any hints - since maybe at some point it would be nice if I could update this machine to 7 as well ;-)
 
Last edited:

jf2021

New Member
Jul 17, 2021
11
4
3
52
Hi ! I've been able to passthrough a PCIe GTX1660 on a fresh install of Proxmox 7 with no problem, as well as many other devices : USB keyboard and mouse, a whole SATA HDD... My MB is a Gigabyte X570

I haven't tried on an upgraded Proxmox but I'd suspect some configuration changes, or device addresses change.

Indeed I noticed some weird things with PCIe card addresses. For example after I physically removed one extra PCI card from the motherboard, the address of the remaining (untouched) GPU had changed in proxmox and therefore it was not working anymore. I had to find the new adress with lspci and update the conf of the VM.

Also I've been having some problems trying to passthrough an old GT640 GPU on Proxmox 7, that works if it's plugged as secondary GPU, but not as first... As you can see on this thread.

Hope this will give you som clues... there are not many answers for that type of questions on this forum unfortunately...
 
  • Like
Reactions: Asano

eminempt

Member
Apr 11, 2021
31
0
6
29
thanks a lot for your insights!! im having problems aswell with that:

I would like to have my Windows VM comming out of my HDMI output on my Graphics card.

Im running PxMox 7 in a ryzen 9 5950x processor, x570 motherboard and a 1650 Nvidia GPU.

Since my CPU does not have Graphics processing unit, i only have one GPU at all. (Nvidia)

so now my question is, would that be the reason why my GPU Passtrought is not working?
 
Last edited:
Jan 27, 2018
45
8
13
41
@eminempt I'm currently not sure if you already had the passthrough working once with Proxmox 6 or if you are trying on a fresh install. Generally speaking I could imagine getting a passthrough to work with one GPU only could be really tricky. For the primary GPU I always needed a ROM file to get passthrough working and here ROM files from the internet (Techpowerup) never worked for me. So I had to extract it myself and that again was only doable while the GPU was *not* the primary which again means at least during that time there should be a second GPU which is the primary. For what it is worth also have a look here: https://forum.proxmox.com/threads/gpu-passthrough-working-for-2-gpu-but-not-for-1.92825/post-411488 but there may be other threads with better info for your case.

Regarding the OT I recently did the in place upgrade from the currently latest 6 to 7 again and now everything worked smooth. Not sure what the actual issue the first time was. I believe in the version back than q35 version pinning was not yet here. However after the upgrade my Windows VMs with passthrough also worked (after some self-adjustments) after setting the q35 version to 6.1...
 
  • Like
Reactions: eminempt

eminempt

Member
Apr 11, 2021
31
0
6
29
hey @Asano.
Check that post you just sent me over there, i managed to solve my problem, and replied there! now working perfecly fine with GPU passtrough. just had to give one argument to the Grub and use a .rom file like you mentioned.

And also, i used one from Techpowerup. i Saw a really insighful video on how to make them work !! :)

here you go:
https://www.youtube.com/watch?v=1IP-h9IKof0 :)

got it working with some tweaks..

also just managed to passtrough the audio devices 1 hour ago.
wasnt working by just adding the device to the vm (throwed me the host into error looping boot state) needed 2 hour TSHing but what was missing was on /etc/modprobe.d/vfio.conf
options vfio-pci ids=10de:1f82,10de:10fa disable_vga=1
options vfio-pci ids=1022:1487 [audio device]


graphics card was a lot harsher, it was working at some point but w/ Error 43. i noticed that didnt occured while using VGA device (VNC) from the browser.. so i could only assume Proxmox HV aswell was trying to use the GPU aswell. Completly disabled it from the Hypervisor perspective at /etc/default/grub by adding "video=efifb:eek:ff"

answering your question about PVE version, i tried on 6.4 when i was not as experienced, and now i managed to do it in 7.1.
what i think your problem could've problably been (read somewhere) related w/ the VM .conf file, or any other thing (might be confused at this point lol) whereas new Proxmox 7.1 already has some parameters on the Kernel, and before you would've had to put them on the .conf or other file, i think.

for what its worth, here is the arguments i put on my .conf file. Problably some are not needed anymore (kvm=off, cpu hidden etc) i dont know. its working.

args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
bios: ovmf
boot: order=sata0
cores: 10
cpu: host,hidden=1



if you need, i've logged all the changes that i did on the Hypervisor! thanks for getting back to me

Marco
 

eminempt

Member
Apr 11, 2021
31
0
6
29
hey one question @Asano, just noticed that you have the same motherboard as i have.
Did you managed to make RGB Fusion working to control the motherboards lights? i guess that would not be easy at all?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!