1 server, 3 GPU's - is it possible?

[KingTChoka]

OK, so currently I have 2 GPU's:

1  NVIDIA GTX 1070
1  AMD Ryzen 5700G (APU w/ integrated graphics).

*** I want to be able to pass the integrated graphics from the AMD CPU to a VM so that VM has access to hardware acceleration. ***

The NVIDIA GPU is already being passed through to another VM, so it's already used and unavailable.

I also want to be able to have monitor access on my server, so I was hoping to buy a very cheap video graphics card to have that as a monitor output display.

Is this set up even possible? It seems pretty complex, so if anyone has a good guide or step-by-step solution, I would be very grateful! Thanks!

My tech stack:

MOTHERBOARD:
B550 AORUS ELITE AX V2

Expansion Slots
1 x PCI Express x16 slot (PCIEX16), integrated in the CPU:
AMD Ryzen™ 5000 Series and Ryzen™ 3000 Series Processors support PCIe 4.0 x16 mode
AMD Ryzen™ Ryzen™ 5000 G-Series and Ryzen™ 4000 G-Series Processors support PCIe 3.0 x16 mode
* For optimum performance, if only one PCI Express graphics card is to be installed, be sure to install it in the PCIEX16 slot.
1 x PCI Express x16 slot (PCIEX2), integrated in the Chipset:
Supporting PCIe 3.0 x2 mode
1 x PCI Express x16 slot (PCIEX1_2), integrated in the Chipset:
Supporting PCIe 3.0 x1 mode
1 x PCI Express x1 slot (PCIEX1_1), integrated in the Chipset:
Supporting PCIe 3.0 x1 mode

Onboard Graphics
Integrated Graphics Processor:
1 x DisplayPort, supporting a maximum resolution of 5120x2880@60 Hz
* Support for DisplayPort 1.4 version, HDCP 2.3, and HDR.
1 x HDMI port, supporting a maximum resolution of 4096x2160@60 Hz
* Support for HDMI 2.1 version, HDCP 2.3, and HDR.
Maximum shared memory of 16 GB

CPU
AMD Ryzen 7 5700G with Radeon Graphics

PROXMOX
Virtual Environment 7.1-4

 

[jdancer]

This may help but then again the GPUs are discrete https://www.youtube.com/watch?v=pIdCV1H1_88

 

[KingTChoka]

This may help but then again the GPUs are discrete https://www.youtube.com/watch?v=pIdCV1H1_88

I've been able to pass through the descrete GPU before, but the new challenge is passing in the integrated graphics from the CPU to a VM.
I believe it's possible with intel CPU's, so I was hoping it would be possible for AMD's as well. I'm just waiting / needing for guidance on that.

There is also the matter of how to install a seperate video graphics card for monitor display and getting proxmox to use that for monitor instead of the CPU's graphics

 

[leesteken]

If you have the right motherboard and the right GPUs, it's not complex. It's just passthrough of three PCIe devices. Not all GPUs reset properly and you might want to make sure that they are known to work with passthrough (by searching for experiences from other people on the internet).

The B550-chipset does not split all on-board devices and PCIe slots into separate IOMMU groups; you need a X570(S) for that. Or use the pcie_acs_override-kernel parameter to "break" the groups but that also means no proper isolation between devices (and no guarantees that it will work).

Passthrough of an integrated GPU is not always possible and often drivers for those integrated don't expect to see the integrated GPU without full access to the CPU and memory. See also my remark about GPUs and resets. Most motherboard don't allow you to choose the boot GPU except for the integrated GPU, which is therefore best used for the Proxmox host.

PCIe devices can work in bigger and smaller slots and with different versions (usually without problems) but performance will typically be reduced. And they all share the limited bandwidth between the chipset and the CPU, unless they use the PCIe lanes coming from the CPU (which is only one PCIe (which is x8 with an APU) and one M.2 on the B550).

 

[jdancer]

I've been able to pass through the descrete GPU before, but the new challenge is passing in the integrated graphics from the CPU to a VM.
I believe it's possible with intel CPU's, so I was hoping it would be possible for AMD's as well. I'm just waiting / needing for guidance on that.

There is also the matter of how to install a seperate video graphics card for monitor display and getting proxmox to use that for monitor instead of the CPU's graphics

This post is for Intel iGPU. Maybe it can give you hints for AMD https://3os.org/infrastructure/proxmox/gpu-passthrough/igpu-passthrough-to-vm/

 

[KingTChoka]

@leesteken
With my motherboard and CPU spec above, is it possible to have a second dedicated GPU to use for a VM? It's main purpose would be fore video decoding. There is an available x16 slot capable of running in x2 mode, which for video decoding should be fine right?

 

[Dunuin]

@leesteken
With my motherboard and CPU spec above, is it possible to have a second dedicated GPU to use for a VM? It's main purpose would be fore video decoding. There is an available x16 slot capable of running in x2 mode, which for video decoding should be fine right?

I think with a B550 you only got one PCIe slot that can be used with PCI passthrough because all other slots are connected to the chipset and not directly to the CPU. You will have to test that, but probably that slot won't get a dedicated IOMMU group.

 

[leesteken]

@leesteken
With my motherboard and CPU spec above, is it possible to have a second dedicated GPU to use for a VM?

The B550-chipset does not split all on-board devices and PCIe slots into separate IOMMU groups; you need a X570(S) for that. Or use the pcie_acs_override-kernel parameter to "break" the groups but that also means no proper isolation between devices (and no guarantees that it will work).

It's main purpose would be fore video decoding. There is an available x16 slot capable of running in x2 mode, which for video decoding should be fine right?

PCIe devices can work in bigger and smaller slots and with different versions (usually without problems) but performance will typically be reduced. And they all share the limited bandwidth between the chipset and the CPU, unless they use the PCIe lanes coming from the CPU (which is only one PCIe (which is x8 with an APU) and one M.2 on the B550).