Finally my nvidia card working with GPU passtrough
Just one step before I would gave up proxmox competelly and try something else
I had two hardvare issues, my motherboard"s BIOS needed to update, and my VGA card BIOS wasn't UEFI compatible and/or was corrupted.
I've run rom parser again to check UEFI compatibility:
root@pve:~# cd rom-parser
root@pve:~/rom-parser# ./rom-parser image.rom
Valid ROM signature found @0h, PCIR offset 190h
PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 128b, class: 030000
PCIR: revision 0, vendor revision: 1
Error, ran off the end
For UEFI support, revision should be 3, and the error message also not looked good.
I've found an unrecommended/supported rom exactly for my card (MSI, 1GB Ram)
https://www.techpowerup.com/vgabios/202493/202493
Originally techpowerup showed me bios only for a 2GB one.
I've tested with rom parser:
root@pve:~/rom-parser# ./rom-parser 202493.rom
Valid ROM signature found @600h, PCIR offset 190h
PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 128b, class: 030000
PCIR: revision 0, vendor revision: 1
Valid ROM signature found @fc00h, PCIR offset 1ch
PCIR: type 3 (EFI), vendor: 10de, device: 128b, class: 030000
PCIR: revision 3, vendor revision: 0
EFI: Signature Valid, Subsystem: Boot, Machine: X64
Last image
Revision 3 seemed promising, and it had no error.
After the motherboard's BIOS update, first I've tried to add vga rom to vm.conf, but it didn't worked.
So I've flashed it to the card:
root@pve:~/rom-parser# ./nvflash_linux 202493.rom
NVIDIA Firmware Update Utility (Version 5.414.0)
Simplified Version For OEM Only
Checking for matches between display adapter(s) and image(s)...
Adapter: GeForce GT 710 (10DE,128B,1462,8C93) H:--:NRM S:00,B:01,D:00,F:00
Current - Version:80.28.A6.00.1F ID:10DE:128B:1462:8C93
GK208 Board - 21320014 (Normal Board)
Replace with - Version:80.28.A6.00.6E ID:10DE:128B:1462:8C93
GK208 Board - 21320014 (Normal Board)
Update display adapter firmware?
Press 'y' to confirm (any other key to abort): y
Identifying EEPROM...
EEPROM ID (7F,9D42) : PMC Pm25LQ020 2.3-3.6V 2048Kx1S, page
Storing updated firmware image...
..........................................
Verifying update...
Update successful.
Firmware image has been updated from version 80.28.A6.00.1F to 80.28.A6.00.6E.
A reboot is required for the update to take effect.
Then I've started vm, but not worked.
So, Ive started looking at my motherboard's BIOS settings again, and I've found, that in Devices -> VGA cards not just my nvidia, but my intel (CPU) also appeared, and (with monitor connected to nvidia hdmi) i can choose between them who would be the primary vga, so I've choosed intel, in that case, nvidia remained for the virtual machines. (onboard vga was enabled and was grayed, so I can't swith it on/off)
I set my vm hardware like: display -> none, I've add nvidia vga with full, pci-express, primary vga, no special rom file, and also I've add USB passtrough for my connected keyboard, and mouse.
Fingers crossed....
Tada!
All three of my vm works fine, with vga support
(ok, ubuntu needed to update nvidia driver for full hd suupport)
Only one step remained: remote acces with headless mode. I've left it for today, Maybe I'll need my hdmi dummy plug.
This is my Windows 10 vm.conf:
agent: 1,type=virtio
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv-vendor-id=NV43FIX,kvm=off'
bios: ovmf
boot: cdn
bootdisk: virtio1
cores: 4
cpu: host,hidden=1,flags=+pcid
cpuunits: 250000
efidisk0: BigData:vm-100-disk-1,size=4M
hostpci0: 01:00,pcie=1,x-vga=1
machine: pc-q35-3.1
memory: 6144
name: Windows10
net0: virtio=42:E8:57:C5:38:10,bridge=vmbr0
numa: 0
ostype: win10
scsihw: virtio-scsi-pci
smbios1: uuid=5a72c8fc-c42f-4030-bd49-85f0e6662e3d
sockets: 1
usb0: host=2-1.7,usb3=1
usb1: host=2-1.8,usb3=1
vga: none
virtio1: BigData:vm-100-disk-0,cache=writeback,size=120G
vmgenid: 4c54e00a-3f82-4c16-bf75-ea7c76b72122
