GPU Passthrough Troubleshooting: Constant BAR 0 Can't Reserve Mem Errors

[planedrop]

Wanted to start off this post that I have done my due diligence, already tried the following tutorials/troubleshooting:

• https://pve.proxmox.com/wiki/Pci_passthrough
• https://manjaro.site/how-to-enable-gpu-passthrough-on-proxmox-6-2/
• https://blog.quindorian.org/2018/03/building-a-2u-amd-ryzen-server-proxmox-gpu-passthrough.html/
• https://www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/
• https://blog.quindorian.org/2018/03/building-a-2u-amd-ryzen-server-proxmox-gpu-passthrough.html/

And I'm getting close but am troubleshooting one last thing. I can boot the VM and ProxMox shows up on the GPU output but it never finished booting into Windows 10.



My dmesg output fills up with:

• vfio-pci 000:09:00.0 (the GPU ID): BAR 0: can't reserve [mem 0xe0000000-0xefffffff 64bit pref]

I'm really not sure where to go from here, I even tried the GPU romfile setting with no luck. Here is all my system and config info:



Specs:

• Ryzen 1600
• 16GB of RAM
• ProxMox 6.4
• AMD RX580 8GB GPU in Primary PCIe slot, it's an ITX system so trying another slot is a no go
• ASRock Motherboard (will need to lookup the model)

BIOS Settings:

• CSM disabled
• IOMMU enabled
• Virtualization enabled

VM Config Settings:









Edit: wanted to show my grub settings as well:







The part after the cutoff above for the default settings.

 

[NerdyNigel]

did you end up solving this? I am having the same issue after a reinstall on a system that used to work fine.

 

[leesteken]

If you updated to pve-kernel-5.15, then you might also need the video=simplefb:off kernel parameter. With the recent kernels, I found that not blacklisting admgpu and not binding the device to vfio-pci for my RX570 works best.
I use vendor-reset to make sure it resets properly after amdgpu has used the GPU during boot of the Proxmox host. Since kernel 5.15, you need to specify a device-specific reset before starting the VM: echo 'device_specific' >"/sys/bus/pci/devices/${PCI_ID_OF_GPU}/reset_method".

 

[rifaterdemsahin]

any timeline for this to be fixed or should I sell my radeon 7 ?

 

[leesteken]

any timeline for this to be fixed or should I sell my radeon 7 ?

View attachment 36412

That does not need fixing. You can now chose which method to use (and it is not part of Proxmox). Before starting the VM run echo 'device_specific' >"/sys/bus/pci/devices/0000:03:00.0/reset_method". Or just run it automatically when starting Proxmox, via cron or /etc/rc.local or whatever works best for you.

 

[rifaterdemsahin]

That does not need fixing. You can now chose which method to use (and it is not part of Proxmox). Before starting the VM run echo 'device_specific' >"/sys/bus/pci/devices/0000:03:00.0/reset_method". Or just run it automatically when starting Proxmox, via cron or /etc/rc.local or whatever works best for you.

thanks added this as cron jobs



how can i check if it works ?

 

[leesteken]

thanks added this as cron jobs
View attachment 36421

That should work for the Radeon VII but you don't need vendor-reset for your RX 6900XT. Better to not change the reset_method for the RX 6900XT (0000:43:00).

how can i check if it works ?

Does the Radeon VII reset properly and work in the VM (even after shutting the VM down and restarting it)? Do you see lines in journalctl -b 0 with AMD_VEGA20 or lines with performing reset?

 

[Mesingel]

In my case, I already had a script in my crontab, which did the following for each PCIe device to be ignored by the host OS:
echo 1 > /sys/bus/pci/devices/0000:03:00.0/remove
Ending with a rescan:
echo 1 > /sys/bus/pci/rescan

I thought to replace the command with the aforementioned one, but you seem to still need both (which makes sense, I suppose?).

Anyways, I mostly wanted to thank leesteken for teaching me the journalctl -b 0 command, which shows the journal logs (if available) starting from a system boot. There's much I need to learn...