Secondary GPU Passthrough Issue

dss_doc

New Member
Dec 29, 2022
2
0
1
I am new to Proxmox, and finding it to be exceptionally good. I have successfully created an Ubuntu VM with my first Nvidia 3090ti GPU passed through (and it works best when the GPU is connected to the monitor). I am having difficulties, however, passing through the second GPU (Nvidia 3080). My setup is using an EFI boot. Looking at the contents of the /proc/iomem file, i see a key difference between the first GPU (IOMMU 41) and the second GPU (IOMMU 61) with the former including a line for vfio-pci. From my naive reading, there does not appear to be any other devices in the IOMMU group, and I have the same selection settings in the GUI (all functions checked; primary GPU checked).

When I reboot the server with my HDMI cable connected to the second GPU, this does not appear to help correct this. Suggestions? thank you!

file contents:
00000000-00000fff : Reserved
00001000-0008ffff : System RAM
00090000-00090fff : Unknown E820 type
00091000-0009ffff : System RAM
000a0000-000fffff : Reserved
000a0000-000bffff : PCI Bus 0000:20
000c0000-000dffff : PCI Bus 0000:00
000f0000-000fffff : System ROM
00100000-03ffffff : System RAM
04000000-04022fff : ACPI Non-volatile Storage
04023000-09df1fff : System RAM
09df2000-09ffffff : Reserved
0a000000-b1edafff : System RAM
b1edb000-b1f20fff : Reserved
b1f21000-b3ca2fff : System RAM
b3ca3000-b3ca3fff : Reserved
b3ca4000-b5bfafff : System RAM
b5bfb000-b8dbefff : Reserved
b8d62018-b8d62019 : APEI ERST
b8d6201c-b8d62021 : APEI ERST
b8d62028-b8d62039 : APEI ERST
b8d62040-b8d6204c : APEI ERST
b8d62050-b8d6404f : APEI ERST
b8da2000-b8da5fff : MSFT0101:00
b8da2000-b8da5fff : MSFT0101:00
b8da6000-b8da9fff : MSFT0101:00
b8da6000-b8da9fff : MSFT0101:00
b8dbf000-b8fe0fff : ACPI Tables
b8fe1000-b972cfff : ACPI Non-volatile Storage
b972d000-bb13ffff : Reserved
bb140000-bb1fefff : Unknown E820 type
bb1ff000-bbffffff : System RAM
bc000000-bfffffff : Reserved
c0000000-c4ffffff : PCI Bus 0000:20
c0000000-c00fffff : PCI Bus 0000:2c
c0000000-c0003fff : 0000:2c:00.0
c0000000-c0003fff : nvme
c0100000-c01fffff : PCI Bus 0000:2d
c0100000-c0103fff : 0000:2d:00.0
c0100000-c0103fff : nvme
c0200000-c04fffff : PCI Bus 0000:2f
c0200000-c02fffff : 0000:2f:00.1
c0200000-c02fffff : ccp
c0300000-c03fffff : 0000:2f:00.3
c0300000-c03fffff : xhci-hcd
c0400000-c0407fff : 0000:2f:00.4
c0408000-c0409fff : 0000:2f:00.1
c0408000-c0409fff : ccp
c5180000-c51fffff : amd_iommu
c5200000-c5200fff : Reserved
c5200000-c52003ff : IOAPIC 3
e0000000-efffffff : PCI MMCONFIG 0000 [bus 00-ff]
e0000000-efffffff : Reserved
e0000000-efffffff : pnp 00:01
f0000000-f10fffff : PCI Bus 0000:40
f0000000-f10fffff : PCI Bus 0000:41
f0000000-f0ffffff : 0000:41:00.0
f0000000-f0ffffff : vfio-pci
f1000000-f107ffff : 0000:41:00.0
f1080000-f1083fff : 0000:41:00.1
f1080000-f1083fff : vfio-pci
f1200000-f127ffff : amd_iommu
f1280000-f1280fff : Reserved
f1280000-f12803ff : IOAPIC 2
f2200000-f22fffff : Reserved
f2200000-f22fffff : pnp 00:00
f2210510-f221053f : MSFT0101:00
f2300000-f237ffff : amd_iommu
f2380000-f2380fff : Reserved
f2380000-f23803ff : IOAPIC 1
f2400000-f24fffff : Reserved
f4000000-f50fffff : PCI Bus 0000:60
f4000000-f50fffff : PCI Bus 0000:61
f4000000-f4ffffff : 0000:61:00.0
f5000000-f507ffff : 0000:61:00.0
f5080000-f5083fff : 0000:61:00.1
f5100000-f51fffff : PCI Bus 0000:00
f5100000-f51fffff : PCI Bus 0000:02
f5100000-f51fffff : 0000:02:00.3
f5100000-f51fffff : xhci-hcd
f5300000-f537ffff : amd_iommu
f5380000-f5380fff : Reserved
f5380000-f53803ff : IOAPIC 4
fea00000-feafffff : Reserved
fec00000-fec00fff : Reserved
fec00000-fec003ff : IOAPIC 0
fec10000-fec10fff : Reserved
fec10000-fec10fff : pnp 00:06
fed00000-fed00fff : Reserved
fed00000-fed003ff : PNP0103:00
fed40000-fed44fff : Reserved
fed80000-fed8ffff : Reserved
fed81500-fed818ff : AMDI0030:00
fedc0000-fedc0fff : pnp 00:06
fedc2000-fedcffff : Reserved
fedc3000-fedc3fff : AMDI0010:01
fedc3000-fedc3fff : AMDI0010:01 AMDI0010:01
fedd4000-fedd5fff : Reserved
fee00000-fee00fff : Local APIC
fee00000-fee00fff : pnp 00:06
ff000000-ffffffff : Reserved
ff000000-ffffffff : pnp 00:06
100000000-803f1fffff : System RAM
30af400000-30b04024c7 : Kernel code
30b0600000-30b0ff8fff : Kernel rodata
30b1000000-30b143cb7f : Kernel data
30b1784000-30b1dfffff : Kernel bss
803f200000-803fffffff : Reserved
10030000000-10041ffffff : PCI Bus 0000:60
10030000000-10041ffffff : PCI Bus 0000:61
10030000000-1003fffffff : 0000:61:00.0
10040000000-10041ffffff : 0000:61:00.0
10080000000-10091ffffff : PCI Bus 0000:40
10080000000-10091ffffff : PCI Bus 0000:41
10080000000-1008fffffff : 0000:41:00.0
10080000000-1008fffffff : vfio-pci
10090000000-10091ffffff : 0000:41:00.0
10090000000-10091ffffff : vfio-pci
100c0400000-100c0cfffff : PCI Bus 0000:20
100c0400000-100c0cfffff : PCI Bus 0000:21
100c0400000-100c0cfffff : PCI Bus 0000:22
100c0400000-100c0cfffff : PCI Bus 0000:25
100c0400000-100c07fffff : 0000:25:00.1
100c0400000-100c07fffff : ixgbe
100c0800000-100c0bfffff : 0000:25:00.0
100c0800000-100c0bfffff : ixgbe
100c0c00000-100c0c03fff : 0000:25:00.1
100c0c00000-100c0c03fff : ixgbe
100c0c04000-100c0c07fff : 0000:25:00.0
100c0c04000-100c0c07fff : ixgbe
 
Last edited:
As an update, I've cloned the working VM using the first GPU and reassigned the second GPU to the VM. Unfortunately, it does not appear to address the issue (attempting to sign in using nomachine simply brings a blank black screen). This is a different behaviour from the original VM which can be accessed by nomachine, but it fails an nvidia-smi command.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!