Another GPU passthrough post (AMD Vega 7)

encrypted_

New Member
Oct 30, 2023
9
0
1
Hello there,

First of all, sorry for posting again about a recurring issue but I've read everything I could and still couldn't make it work.
My system:

Asrock A320M-HDR R4
Ryzen 4650Pro
32GB ECC UDIMM
I got some success following this video tutorial: https://www.bilibili.com/video/BV11d4y1G7Nk/

What I see:
  • Each pci component in correctly in it's iommu group.
  • I was able to boot the VM with both VIRTIO GPU display and PCIE device
  • I was able to install the AMD drivers
  • When I removed the VIRTIO GPU display and selected None, keeping the PCIE it won't boot.
  • I can boot the VM when the PCIE line is not present
  • The VM won't boot after I add the PCI line

/etc/default/grub
Code:
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt amd_iommu=on pcie_acs_override=downstream,multifunction initcall_blacklist-sysfb_init"
GRUB_CMDLINE_LINUX=""


/etc/modprobe.d/blacklist.conf
Code:
blacklist amdgpu
blacklist snd_hda_intel

/etc/modprobe.d/kvm.conf
Code:
options kvm ignore_msrs=1

/etc/modprobe.d/vfio.conf
Code:
options vfio-pci ids=1002:1636,1002:1637
softdep radeon pre: vfio-pci
softdep amdgpu pre: vfio-pci

/etc/modules
Code:
jc42
nct6775
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
vendor-reset

/etc/pve/qemu-server/103.conf
Code:
bios: seabios
boot: order=virtio0;ide2;net0;ide0
cores: 4
cpu: host,hidden=1
hostpci0: 0000:06:00,pcie=1,romfile=vbios_1636.dat,x-vga=1
ide0: local:iso/virtio-win-0.1.229.iso,media=cdrom,size=522284K
ide2: local:iso/Win10_22H2_BrazilianPortuguese_x64.iso,media=cdrom,size=5666236K
machine: pc-q35-7.2
memory: 8192
meta: creation-qemu=7.2.0,ctime=1698553568
name: win10-igpu
net0: virtio=E6:48:0A:D8:23:10,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
scsihw: virtio-scsi-single
smbios1: uuid=b725946b-3fd3-4f8e-aff2-ceb78533f4b0
sockets: 1
vga: qxl
virtio0: local-lvm:vm-103-disk-0,iothread=1,size=32G
vmgenid: a9ab640a-43de-409f-bd32-65d7c7d4c239

dmesg | grep -e DMAR -e IOMMU
Code:
[    0.000000] Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA
[    0.369860] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.370438] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    0.689802] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).


cat /proc/cmdline; for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done

Code:
BOOT_IMAGE=/boot/vmlinuz-5.15.116-1-pve root=/dev/mapper/pve-root ro quiet iommu=pt amd_iommu=on pcie_acs_override=downstream,multifunction initcall_blacklist=sysfb_init
IOMMU group 0 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU group 10 01:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b3] (rev 02)
IOMMU group 11 02:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU group 12 02:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU group 13 02:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU group 14 04:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 04)
IOMMU group 15 05:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)
IOMMU group 16 06:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Renoir [1002:1636] (rev ff)
IOMMU group 17 06:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:1637] (rev ff)
IOMMU group 18 06:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor [1022:15df] (rev ff)
IOMMU group 19 06:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Renoir USB 3.1 [1022:1639] (rev ff)
IOMMU group 1 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU group 20 06:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Renoir USB 3.1 [1022:1639] (rev ff)
IOMMU group 21 06:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) HD Audio Controller [1022:15e3] (rev ff)
IOMMU group 22 07:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 81)
IOMMU group 23 07:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 81)
IOMMU group 2 00:02.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe GPP Bridge [1022:1634]
IOMMU group 3 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU group 4 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus [1022:1635]
IOMMU group 5 00:08.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus [1022:1635]
IOMMU group 6 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 51)
IOMMU group 6 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU group 7 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir Device 24: Function 0 [1022:1448]
IOMMU group 7 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir Device 24: Function 1 [1022:1449]
IOMMU group 7 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir Device 24: Function 2 [1022:144a]
IOMMU group 7 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir Device 24: Function 3 [1022:144b]
IOMMU group 7 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir Device 24: Function 4 [1022:144c]
IOMMU group 7 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir Device 24: Function 5 [1022:144d]
IOMMU group 7 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir Device 24: Function 6 [1022:144e]
IOMMU group 7 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir Device 24: Function 7 [1022:144f]
IOMMU group 8 01:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:43bc] (rev 02)
IOMMU group 9 01:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b8] (rev 02)

lspci -nnv
Code:
see attached
 

Attachments

  • lspci-nnv.txt
    16.4 KB · Views: 2
Last edited:
So I managed to make it work but only once each host reset.
If I reboot the VM it's stops working. Still looking for a solution.
 
Unfortunatelly my iGPU (Vega 7 from Ryzen 4650G Pro) is not supported by vendor-reset.
My settings are exactly as above. It boots with those configuration when host is reseted but as I said, only once.
 
Unfortunatelly my iGPU (Vega 7 from Ryzen 4650G Pro) is not supported by vendor-reset.
My settings are exactly as above. It boots with those configuration when host is reseted but as I said, only once.
Have you tried to reboot your Proxmox host?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!