Add GPU to Windows VM

[TheEnigmist]

Hello,
I'm able to run a Windows 11 24H2 VM without any problems using host CPU passthrough, but now I'm having trouble adding a GPU to the VM.
My current system is:
CPU: Ryzen 5 3400G
GPU: GeForce GT 710 (installed to free up the AMD GPU)

I have enabled IOMMU in the BIOS, and installed virtio on guest machine, but now I don't know how to assign the AMD GPU to my VM.
These are my current VM settings:

Shoul I edit Display settings?

 

[cheiss]

Please see also the PCI(e) Passthrough wiki page, if you haven't yet

After that, to add it to the VM, you'll need to select Add > PCI device and then either select your resource mapping or directly the PCI device (your AMD GPU) to pass through to the VM.

 

[TheEnigmist]

OK, I’ve read the wiki page and performed the checks for IOMMU—everything passed successfully. I also listed all the IOMMU groups, but I can see only one VGA device (the GT 710).
I'm certain that the host output is running from the GT 710, so the 3400G GPU should be available. However, I don't see it in the list, should be 00:00.2 IOMMU? Am I missing something?

00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Root Complex
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 IOMMU
00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0]
00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0]
00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Internal PCIe GPP Bridge 0 to Bus A
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 61)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 3
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 5
00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 6
00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 7
01:00.0 VGA compatible controller: NVIDIA Corporation GK208B [GeForce GT 710] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GK208 HDMI/DP Audio Controller (rev a1)
02:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset USB 3.1 xHCI Controller (rev 02)
02:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset SATA Controller (rev 02)
02:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b2 (rev 02)
03:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
03:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
03:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
03:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
03:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
03:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
08:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)
09:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 11)
0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function (rev c8)
0a:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor
0a:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Raven USB 3.1
0a:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Raven USB 3.1
0a:00.6 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h/19h HD Audio Controller

 

[leesteken]

I also listed all the IOMMU groups, but I can see only one VGA device (the GT 710).
I'm certain that the host output is running from the GT 710, so the 3400G GPU should be available. However, I don't see it in the list, should be 00:00.2 IOMMU? Am I missing something?

Maybe it's (automatically) disabled by the motherboard BIOS since you selected a discrete GPU? The integrated GPU will be in a big IOMMU group and passthrough will requires work-arounds and maybe a patched ROM-file since it's integrated graphics (search the forum for your exact CPU/GPU). It might be easier to boot the system with the integrated AMD graphics and passthrough the discrete NVidia GPU (if it's in the first x16 PCIe slot connected to the CPU that is in its own IOMMU group).

 

[Impact]

The iGPU might be disabled in the UEFI.
In my case I had to enable it via Advanced > AMD CBS > NBIO Common Options > GFX Configuration > iGPU Configuration > UMA_SPECIFIED.

 

[TheEnigmist]

Maybe it's (automatically) disabled by the motherboard BIOS since you selected a discrete GPU? The integrated GPU will be in a big IOMMU group and passthrough will requires work-arounds and maybe a patched ROM-file since it's integrated graphics (search the forum for your exact CPU/GPU). It might be easier to boot the system with the integrated AMD graphics and passthrough the discrete NVidia GPU (if it's in the first x16 PCIe slot connected to the CPU that is in its own IOMMU group).

The GT 710 is less powerful than the integrated AMD GPU. I only installed it so the host could use it for basic shell and graphical tasks, leaving the integrated GPU available for passthrough. I'm aware there are some known bugs and workarounds regarding Ryzen CPUs and integrated GPUs, so I thought that giving the discrete GPU to Proxmox would simplify passing the integrated GPU through to the guest VM.

The iGPU might be disabled in the UEFI.
In my case I had to enable it via Advanced > AMD CBS > NBIO Common Options > GFX Configuration > iGPU Configuration > UMA_SPECIFIED.

I'll check this, thanks.

 

[TheEnigmist]

The iGPU might be disabled in the UEFI.
In my case I had to enable it via Advanced > AMD CBS > NBIO Common Options > GFX Configuration > iGPU Configuration > UMA_SPECIFIED.

This was the issue; now it appears in the IOMMU list, but if I use it directly as a PCIe device, my entire system reboots after many AMD-Vi timeout messages (I checked them on a monitor attached to the guest system)

 

[leesteken]

This was the issue; now it appears in the IOMMU list, but if I use it directly as a PCIe device, my entire system reboots after many AMD-Vi timeout messages (I checked them on a monitor attached to the guest system)

The integrated GPU will be in a big IOMMU group and passthrough will requires work-arounds and maybe a patched ROM-file since it's integrated graphics (search the forum for your exact CPU/GPU).

You probably need to break your IOMMU groups (not recommended) with pcie_acs_override and make of find a patched ROM for your specific integrated graphics. You can search for more information on this forum (and other places). I don't have personal experience with this so I cannot really help any further.

 

[Impact]

@TheEnigmist Can you share the output of this?

#!/bin/bash
shopt -s nullglob
for g in $(find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V); do
    echo "IOMMU Group ${g##*/}:"
    for d in $g/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done;
done;

Also make sure to enable IOMMU/ACS related options in the UEFI. In my case the iGPU has its own group.