[SOLVED] NVIDIA GPU passtrough issues

no you can do also with only a card
I have had with a perfect working system with a passed Nvidia TitanXp card with windows VM
Audio DP/hdmi working perfectly , also my internal motherboard Audio working perfectly (windows VM)
In OSX VM internal audio is not working well (scratch and other problem), Nvidia Dp/hdmi audio is perfect instead (also in OSX VM)
I have followed Proxmox WIKi which is the best resource to follow imho
 
Ok, could we go trough settings?

grub:

Code:
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_acs_override=downstream,multifunction video=efifb:eek:ff"
GRUB_CMDLINE_LINUX=""

modules:

Code:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

/etc/modprobe.d/iommu_unsafe_interrupts.conf
options vfio_iommu_type1 allow_unsafe_interrupts=1

/etc/modprobe.d/kvm.conf
options kvm ignore_msrs=1

blacklist.conf
Code:
blacklist nouveau
blacklist nvidia
blacklist nvidiafb
blacklist snd_hda_codec_hdmi
blacklist snd_hda_intel
blacklist snd_hda_codec
blacklist snd_hda_core
blacklist radeon
blacklist amdgpu

/etc/modprobe.d/vfio.conf
options vfio-pci ids=10de:128b,10de:0e0f

vm config
Code:
agent: 1
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv-vendor-id=nvidia,kvm=off'
bios: ovmf
boot: cdn
bootdisk: virtio1
cores: 4
cpu: host,hidden=1,flags=+pcid
cpuunits: 250000
efidisk0: BigData:vm-100-disk-1,size=4M
hostpci0: 01:00,pcie=on,x-vga=on,romfile=vbios.bin
machine: q35
memory: 6144
name: Windows10
net0: virtio=42:E8:57:C5:38:10,bridge=vmbr0
numa: 0
ostype: win10
scsihw: virtio-scsi-pci
smbios1: uuid=5a72c8fc-c42f-4030-bd49-85f0e6662e3d
sockets: 1
vga: std
virtio1: BigData:vm-100-disk-0,cache=writeback,size=120G
vmgenid: 4c54e00a-3f82-4c16-bf75-ea7c76b72122
 
I can't install vm WITH added PCI device, cause it crash proxmox. Is it normal? Should I have to add PCI video card AFTER vm install?
 
Finally my nvidia card working with GPU passtrough :)
Just one step before I would gave up proxmox competelly and try something else :)

I had two hardvare issues, my motherboard"s BIOS needed to update, and my VGA card BIOS wasn't UEFI compatible and/or was corrupted.

I've run rom parser again to check UEFI compatibility:

root@pve:~# cd rom-parser
root@pve:~/rom-parser# ./rom-parser image.rom
Valid ROM signature found @0h, PCIR offset 190h
PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 128b, class: 030000
PCIR: revision 0, vendor revision: 1
Error, ran off the end

For UEFI support, revision should be 3, and the error message also not looked good.

I've found an unrecommended/supported rom exactly for my card (MSI, 1GB Ram) https://www.techpowerup.com/vgabios/202493/202493
Originally techpowerup showed me bios only for a 2GB one.

I've tested with rom parser:

root@pve:~/rom-parser# ./rom-parser 202493.rom
Valid ROM signature found @600h, PCIR offset 190h
PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 128b, class: 030000
PCIR: revision 0, vendor revision: 1
Valid ROM signature found @fc00h, PCIR offset 1ch
PCIR: type 3 (EFI), vendor: 10de, device: 128b, class: 030000
PCIR: revision 3, vendor revision: 0
EFI: Signature Valid, Subsystem: Boot, Machine: X64
Last image

Revision 3 seemed promising, and it had no error.

After the motherboard's BIOS update, first I've tried to add vga rom to vm.conf, but it didn't worked.

So I've flashed it to the card:

root@pve:~/rom-parser# ./nvflash_linux 202493.rom

NVIDIA Firmware Update Utility (Version 5.414.0)
Simplified Version For OEM Only
Checking for matches between display adapter(s) and image(s)...

Adapter: GeForce GT 710 (10DE,128B,1462,8C93) H:--:NRM S:00,B:01,D:00,F:00

Current - Version:80.28.A6.00.1F ID:10DE:128B:1462:8C93
GK208 Board - 21320014 (Normal Board)
Replace with - Version:80.28.A6.00.6E ID:10DE:128B:1462:8C93
GK208 Board - 21320014 (Normal Board)

Update display adapter firmware?
Press 'y' to confirm (any other key to abort): y
Identifying EEPROM...
EEPROM ID (7F,9D42) : PMC Pm25LQ020 2.3-3.6V 2048Kx1S, page
Storing updated firmware image...
..........................................
Verifying update...
Update successful.

Firmware image has been updated from version 80.28.A6.00.1F to 80.28.A6.00.6E.

A reboot is required for the update to take effect.

Then I've started vm, but not worked.

So, Ive started looking at my motherboard's BIOS settings again, and I've found, that in Devices -> VGA cards not just my nvidia, but my intel (CPU) also appeared, and (with monitor connected to nvidia hdmi) i can choose between them who would be the primary vga, so I've choosed intel, in that case, nvidia remained for the virtual machines. (onboard vga was enabled and was grayed, so I can't swith it on/off)

I set my vm hardware like: display -> none, I've add nvidia vga with full, pci-express, primary vga, no special rom file, and also I've add USB passtrough for my connected keyboard, and mouse.

Fingers crossed....

Tada!

All three of my vm works fine, with vga support :) (ok, ubuntu needed to update nvidia driver for full hd suupport)

Only one step remained: remote acces with headless mode. I've left it for today, Maybe I'll need my hdmi dummy plug.

This is my Windows 10 vm.conf:

agent: 1,type=virtio
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv-vendor-id=NV43FIX,kvm=off'
bios: ovmf
boot: cdn
bootdisk: virtio1
cores: 4
cpu: host,hidden=1,flags=+pcid
cpuunits: 250000
efidisk0: BigData:vm-100-disk-1,size=4M
hostpci0: 01:00,pcie=1,x-vga=1
machine: pc-q35-3.1
memory: 6144
name: Windows10
net0: virtio=42:E8:57:C5:38:10,bridge=vmbr0
numa: 0
ostype: win10
scsihw: virtio-scsi-pci
smbios1: uuid=5a72c8fc-c42f-4030-bd49-85f0e6662e3d
sockets: 1
usb0: host=2-1.7,usb3=1
usb1: host=2-1.8,usb3=1
vga: none
virtio1: BigData:vm-100-disk-0,cache=writeback,size=120G
vmgenid: 4c54e00a-3f82-4c16-bf75-ea7c76b72122
 
  • Like
Reactions: elurex

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!