[SOLVED] Proxmox VE RTX 4060 GPU Passthrough Issues

vineet

Member
Jan 15, 2023
5
2
8
For last one week i have been trying to passthrough RTX 4060 but failed. But i don't have any issues passing Quadro P2000 GPU only have an issue with RTX 4060

My Current attempt is to have RTX 4060 working inside VM but i get the errors on proxmox main display (Quadro P2000) which is still connected.
Here are The Screenshot of the errors i am seeing


Specs:
CPU: AMD EPYC Milan 7C13
RAM: 256 GB
GPU: 1x Quadro P2000 ( Main Display, but i can also passthrough this without any issue ), 1x Zotac RTX 4060 SOLO (Failing to passthrough this GPU)
PSU: ASUS ROG 1000W PSU
Storage: 2x Samsung 256gb ssd (ZFS RAID 1 where os and vm are running)

I have followed multiple guide so see which if any might help:
1. First Guide for GPU Passthrough. This is also my Current active one.
2. Old one i tried to use custom ROM file for GPU but this also doent work for rtx 4060

Proxmox Setup:

Proxmox VE 8.3.0

Grub: GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt nomodeset initcall_blacklist=sysfb_init"
lspci -nnk:

Code:
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation AD107 [GeForce RTX 4060] [10de:2882] (rev a1)

        Subsystem: ASUSTeK Computer Inc. AD107 [GeForce RTX 4060] [1043:8903]

        Kernel driver in use: vfio-pci

        Kernel modules: nvidiafb, nouveau

01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:22be] (rev a1)

        Subsystem: ASUSTeK Computer Inc. Device [1043:8903]

        Kernel driver in use: vfio-pci

        Kernel modules: snd_hda_intel



81:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP106GL [Quadro P2000] [10de:1c30] (rev a1)

        Subsystem: Dell GP106GL [Quadro P2000] [1028:11b3]

        Kernel modules: nvidiafb, nouveau

81:00.1 Audio device [0403]: NVIDIA Corporation GP106 High Definition Audio Controller [10de:10f1] (rev a1)

        Subsystem: Dell GP106 High Definition Audio Controller [1028:11b3]

        Kernel driver in use: snd_hda_intel

        Kernel modules: snd_hda_intel

My VM conf File:
Code:
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
balloon: 0
bios: ovmf
boot: order=scsi0;ide2;net0
cores: 4
cpu: host
hostpci0: 0000:01:00,pcie=1
ide2: local:iso/ubuntu-24.04.2-live-server-amd64.iso,media=cdrom,size=3137758K
machine: q35
memory: 8192
meta: creation-qemu=9.0.2,ctime=1740944404
name: 1
net0: virtio=BC:24:11:C2:03:D7,bridge=vmbr0
numa: 0
ostype: l26
scsi0: local-zfs:vm-100-disk-0,iothread=1,size=32G
scsihw: virtio-scsi-single
smbios1: uuid=23f765af-d665-491b-95d6-e84fb5f67402
sockets: 1
vga: virtio
vmgenid: 2c7e6448-db48-41d3-b8e3-a4a8b5086204

Error:
Code:
root@pve:~# dmesg | grep vfio
[    8.220193] vfio-pci 0000:01:00.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=io+mem:owns=none
[    8.220393] vfio_pci: add [10de:2882[ffffffff:ffffffff]] class 0x000000/00000000
[    8.271690] vfio_pci: add [10de:22be[ffffffff:ffffffff]] class 0x000000/00000000
[   94.548512] vfio-pci 0000:01:00.0: AER: aer_status: 0x000000c1, aer_mask: 0x00000000
[   94.548703] vfio-pci 0000:01:00.0:    [ 0] RxErr                  (First)
[   94.548898] vfio-pci 0000:01:00.0:    [ 6] BadTLP              
[   94.549084] vfio-pci 0000:01:00.0:    [ 7] BadDLLP              
[   94.549267] vfio-pci 0000:01:00.0: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[   94.549474] vfio-pci 0000:01:00.1: AER: aer_status: 0x000000c1, aer_mask: 0x00000000
[   94.549657] vfio-pci 0000:01:00.1:    [ 0] RxErr                  (First)
[   94.549846] vfio-pci 0000:01:00.1:    [ 6] BadTLP              
[   94.550030] vfio-pci 0000:01:00.1:    [ 7] BadDLLP              
[   94.550209] vfio-pci 0000:01:00.1: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[  107.908010] vfio-pci 0000:01:00.0: vfio_bar_restore: reset recovery - restoring BARs

Please feel free to ask me anything else if you want to have better clarity.
 
Last edited:
An Update i tried a RTX 4080 Super from Zotac GPU instead of RTX 4060 Zotac Solo and i was able to passthroughs without doing any configurations to the proxmox(fresh install no config done) so i guess it all boils down to the luck one has based on GPU Model and Vendor. Specially in the case of consumer gpus. Also i feel in consumer GPUs its mostly the higher end ones of the stack that have easier time for gpu passthrough coz i tried passthrough without doing any configurations to proxmox VE

RTX 4080 Super (Zotac) - Worked
RTX 3080 (Inno3d) - Worked
GTX 1080 (GALAX) - worked
Quadro P2000 (Dell) - Worked
GTX 980ti (Zotac) - Worked

The only one that didnt work was RTX 4060 from zotac.

If anyone out there has more info on this please shoot back at me with more clarifications would love to know more