[SOLVED] RTX 4090 Passthrough : AMD, Proxmox and sadness

comma

New Member
Aug 23, 2023
1
1
1
Hi everyone,

I'm losing my sanity a bit on this topic and I would like your previous help, let's begin!


# lspci -vv and info on the device I want to passthrough
0f:00.0 VGA compatible controller: NVIDIA Corporation AD102 [GeForce RTX 4090] (rev a1) (prog-if 00 [VGA controller])
Kernel modules: nvidiafb, nouveau

0f:00.1 Audio device: NVIDIA Corporation AD102 High Definition Audio Controller (rev a1)
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel

lspci -nnv | grep VGA
================
03:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK208B [GeForce GT 710] [10de:128b] (rev a1) (prog-if 00 [VGA controller])
0f:00.0 VGA compatible controller [0300]: NVIDIA Corporation AD102 [GeForce RTX 4090] [10de:2684] (rev a1) (prog-if 00 [VGA controller])

dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
================================================
[ 0.276619] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0
[ 0.714320] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[ 0.718366] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.718368] AMD-Vi: Extended features (0x58f77ef22294a5a, 0x0): PPR NX GT IA PC GA_vAPIC
[ 0.718371] AMD-Vi: Interrupt remapping enabled
[ 0.718591] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).

dmesg | grep 'remapping'
====================
[ 0.442503] x2apic: IRQ remapping doesn't support X2APIC mode
[ 0.718371] AMD-Vi: Interrupt remapping enabled

cat /etc/modules
==============
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
# Parameters can be specified after the module name.
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia*" >> /etc/modprobe.d/blacklist.conf
echo options vfio-pci ids=10de:2684,10de:22ba disable_vga=1 > /etc/modprobe.d/vfio.conf


01:00.0 VGA compatible controller: NVIDIA Corporation Device 2684 (rev a1) (prog-if 00 [VGA controller])
Subsystem: NVIDIA Corporation Device 165b
Physical Slot: 0
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Capabilities: [600 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia

I passed the pcie as per the doc and installed Nvidia drivers from Nvidia, no nvidia-smi available but the driver seems there, also the gpu name doesn't appear on lspci but only the code :)

FIXED

It was secure boot on the ubuntu VM :)

Just disable it and it works :)

 
Last edited:
  • Like
Reactions: rsinghal2000

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!