New GPU not showing in lspci and cannot be used for PCI pass through

[yarfaridkhan]

Hi,

I have a running prox mox environment in my home lab. I recently bought an off the shelf RTX 4060TI and installed it in my machine. Now I want to add this GPU in one of my VM as PCI passthrough. However I can't seems to find in my IOMMU devices. Please let me know if I am missing anything.

I am using a desktop with AMD Ryzen 9 7950X processor with 64 GB RAM and currently running at 8.3.2 version.

The output of lspci is as below

root@service:~# lspci
00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Root Complex
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge IOMMU
00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Dummy Host Bridge
00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Dummy Host Bridge
00:02.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge GPP Bridge
00:02.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge GPP Bridge
00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Dummy Host Bridge
00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Dummy Host Bridge
00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Dummy Host Bridge
00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Internal GPP Bridge to Bus [C:A]
00:08.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Internal GPP Bridge to Bus [C:A]
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 71)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 3
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 5
00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 6
00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge Data Fabric; Function 7
01:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Upstream Port (rev 01)
02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port (rev 01)
02:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port (rev 01)
02:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port (rev 01)
02:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port (rev 01)
02:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port (rev 01)
02:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port (rev 01)
02:0c.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port (rev 01)
02:0d.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port (rev 01)
06:00.0 Network controller: MEDIATEK Corp. MT7921K (RZ608) Wi-Fi 6E 80MHz
07:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
08:00.0 Non-Volatile memory controller: Kingston Technology Company, Inc. NV2 NVMe SSD [SM2267XT] (DRAM-less) (rev 03)
09:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset USB 3.2 Controller (rev 01)
0a:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset SATA Controller (rev 01)
0b:00.0 Non-Volatile memory controller: Kingston Technology Company, Inc. NV2 NVMe SSD [SM2267XT] (DRAM-less) (rev 03)
0c:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Raphael (rev c1)
0c:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Rembrandt Radeon High Definition Audio Controller
0c:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] VanGogh PSP/CCP
0c:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge USB 3.1 xHCI
0c:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge USB 3.1 xHCI
0c:00.6 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h/19h HD Audio Controller
0d:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 15b8

IOMMU devices are on in the server.

root@service:~# dmesg | grep -i iommu
[    0.000000] Command line: initrd=\EFI\proxmox\6.8.12-5-pve\initrd.img-6.8.12-5-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs amd_iommu=on
[    0.045277] Kernel command line: initrd=\EFI\proxmox\6.8.12-5-pve\initrd.img-6.8.12-5-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs amd_iommu=on
[    0.388061] iommu: Default domain type: Translated
[    0.388061] iommu: DMA domain TLB invalidation policy: lazy mode
[    0.415732] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.415767] pci 0000:00:01.0: Adding to iommu group 0
[    0.415791] pci 0000:00:02.0: Adding to iommu group 1
[    0.415807] pci 0000:00:02.1: Adding to iommu group 2
[    0.415822] pci 0000:00:02.2: Adding to iommu group 3
[    0.415840] pci 0000:00:03.0: Adding to iommu group 4
[    0.415858] pci 0000:00:04.0: Adding to iommu group 5
[    0.415880] pci 0000:00:08.0: Adding to iommu group 6
[    0.415895] pci 0000:00:08.1: Adding to iommu group 7
[    0.415910] pci 0000:00:08.3: Adding to iommu group 8
[    0.415936] pci 0000:00:14.0: Adding to iommu group 9
[    0.415950] pci 0000:00:14.3: Adding to iommu group 9
[    0.416010] pci 0000:00:18.0: Adding to iommu group 10
[    0.416024] pci 0000:00:18.1: Adding to iommu group 10
[    0.416037] pci 0000:00:18.2: Adding to iommu group 10
[    0.416050] pci 0000:00:18.3: Adding to iommu group 10
[    0.416063] pci 0000:00:18.4: Adding to iommu group 10
[    0.416076] pci 0000:00:18.5: Adding to iommu group 10
[    0.416089] pci 0000:00:18.6: Adding to iommu group 10
[    0.416103] pci 0000:00:18.7: Adding to iommu group 10
[    0.416119] pci 0000:01:00.0: Adding to iommu group 11
[    0.416144] pci 0000:02:00.0: Adding to iommu group 12
[    0.416164] pci 0000:02:04.0: Adding to iommu group 13
[    0.416185] pci 0000:02:05.0: Adding to iommu group 14
[    0.416205] pci 0000:02:06.0: Adding to iommu group 15
[    0.416226] pci 0000:02:07.0: Adding to iommu group 16
[    0.416247] pci 0000:02:08.0: Adding to iommu group 17
[    0.416267] pci 0000:02:0c.0: Adding to iommu group 18
[    0.416287] pci 0000:02:0d.0: Adding to iommu group 19
[    0.416297] pci 0000:06:00.0: Adding to iommu group 15
[    0.416308] pci 0000:07:00.0: Adding to iommu group 16
[    0.416318] pci 0000:08:00.0: Adding to iommu group 17
[    0.416325] pci 0000:09:00.0: Adding to iommu group 18
[    0.416331] pci 0000:0a:00.0: Adding to iommu group 19
[    0.416350] pci 0000:0b:00.0: Adding to iommu group 20
[    0.416372] pci 0000:0c:00.0: Adding to iommu group 21
[    0.416389] pci 0000:0c:00.1: Adding to iommu group 22
[    0.416406] pci 0000:0c:00.2: Adding to iommu group 23
[    0.416423] pci 0000:0c:00.3: Adding to iommu group 24
[    0.416439] pci 0000:0c:00.4: Adding to iommu group 25
[    0.416456] pci 0000:0c:00.6: Adding to iommu group 26
[    0.416471] pci 0000:0d:00.0: Adding to iommu group 27
[    0.445306] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).