AMD APU (gpu) passthrough

gingi

New Member
Mar 31, 2022
4
0
1
Hello - Relative noob here. I am trying to passthrough my gpu (part of the APU) to a linux vm. I have followed the tutorial here and various permutations and I can't get it to work. Every time I start the VM it causes the host to freeze. Any ssh session is frozen and the web ui is basically frozen (I can click but many things a greyed out and I just see that loading circle icon). I have also tried many different settings in my bios.

Here is system specs:
CPU: AMD Ryzen 7 5700g
MB: MSI B550M VC Wifi
- IOMMU: Enabled
- SR-IOV: Enabled
RAM: 32 gb

Bash:
$ lspci -nnk
[...]
30:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [Radeon Vega Series / Radeon Vega Mobile Series] [1002:1638] (rev c8)
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [Radeon Vega Series / Radeon Vega Mobile Series] [1002:1636]
Kernel driver in use: vfio-pci
Kernel modules: amdgpu
30:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Renoir Radeon High Definition Audio Controller [1002:1637]
Subsystem: Micro-Star International Co., Ltd. [MSI] Renoir Radeon High Definition Audio Controller [1462:9c95]
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
[...]


Bash:
$ cat /etc/modprobe.d/blacklist.conf

blacklist radeon
blacklist amdgpu

Bash:
$ cat /etc/modprobe.d/vfio.conf

options vfio-pci ids=1002:1638,1002:1637

Bash:
$ cat /etc/default/grub
[...]
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt nomodeset initcall_blacklist=sysfb_init"
[...]

Bash:
$ dmesg | grep 'remapping'
[    0.776356] AMD-Vi: Interrupt remapping enabled

Bash:
$ dmesg | grep -E "DMAR|IOMMU"
[    0.775501] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.776340] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    0.833806] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).

Bash:
$ dmesg | grep -i vfio
[    7.269840] VFIO - User Level meta-driver version: 0.3
[    7.325403] vfio-pci 0000:30:00.0: vgaarb: deactivate vga console
[    7.325408] vfio-pci 0000:30:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
[    7.325604] vfio_pci: add [1002:1638[ffffffff:ffffffff]] class 0x000000/00000000
[    7.348471] vfio_pci: add [1002:1637[ffffffff:ffffffff]] class 0x000000/00000000

Bash:
$ cat /etc/modules

vfio
vfio_iommu_type1
vfio_pci


I then add the gpu device to my VM like so:
Screenshot 2023-12-28 at 9.52.25 PM.png
 
Yes. Though it wasn't pretty. You need to get the motherboard bios then I followed a mix of instructions from https://winraid.level1techs.com/t/tool-guide-news-uefi-bios-updater-ubu/30357 and https://www.reddit.com/r/homelab/comments/11l0s5j/boxmox_asrock_4x4_box_5800u_jbod_proxmox/?rdt=5185. You need to add that vbois as options in your .conf

Like
hostpci0: 0000:30:00.0,romfile=vbios_1638.dat,pcie=1,x-vga=1
that's awesome...did you not make any notes step by step???
 
that's awesome...did you not make any notes step by step???
x-vga=1 (Primary GPU) is a work-around for NVidia GPUs and probably not needed. You probably do need to "breaK" the iOMMU groups (with pcie_acs_override, use at your own risk) because the integrated graphics is not properly isolated. Maybe you need additional work-arounds if the integrated graphics does not reset properly.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!