GPU passthrough win10 code 43

[Chapp]

Hi today i tried to install and win10 VM with GPU passthrough but when i install driver the vm crash instant i've tried many config but no one worked
my config :

• I5-9600kf
• gtx960
• ssd 850

cat /proc/cmdline

BOOT_IMAGE=/boot/vmlinuz-5.15.102-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt kvm.ignore_msrs=1 video=vesafb:off video=efifb:off intremap=no_x2apic_optout

 

[leesteken]

BOOT_IMAGE=/boot/vmlinuz-5.15.102-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt kvm.ignore_msrs=1 video=vesafb:off video=efifb:off intremap=no_x2apic_optout

video=vesafb:off video=efifb:off does nothing since July 2019 or so. You need this work-around when doing PCIe passthrough of the boot GPU.
For some GPUs you need additional VM settings, but I don't see the output of qm config YOUR_VM_NUMBER. Do you really need intremap=no_x2apic_optout?

 

[Chapp]

qm config 103

agent: 1
balloon: 0
bios: ovmf
boot: order=scsi0;net0
cores: 6
efidisk0: Samsung:vm-103-disk-0,efitype=4m,pre-enrolled-keys=1,size=4M
hostpci0: 0000:01:00,pcie=1,x-vga=1
machine: pc-q35-7.2
memory: 16384
meta: creation-qemu=7.2.0,ctime=1680763804
name: Win10
net0: e1000=FA:40:68:18:47:C9,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
scsi0: Samsung:vm-103-disk-1,cache=writeback,iothread=1,size=100G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=9be3b909-bcde-41dd-892f-16c4c0176889
sockets: 1
vga: none
vmgenid: 909882c6-0510-4757-b9cc-7b96a02c1a3e

BOOT_IMAGE=/boot/vmlinuz-5.15.102-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt kvm.ignore_msrs=1 video=vesafb:off video=efifb:off intremap=no_x2apic_optout

Honestly i don't know

 

[leesteken]

agent: 1 balloon: 0 bios: ovmf boot: order=scsi0;net0 cores: 6 efidisk0: Samsung:vm-103-disk-0,efitype=4m,pre-enrolled-keys=1,size=4M hostpci0: 0000:01:00,pcie=1,x-vga=1 machine: pc-q35-7.2

In my experience, Windows drivers work better with pc-q35-6.2 instead of 7.2. Try setting Machine Version to 6.2. Keep Primary GPU enabled for NVidia GPUs.

BOOT_IMAGE=/boot/vmlinuz-5.15.102-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt kvm.ignore_msrs=1 video=vesafb:off video=efifb:off intremap=no_x2apic_optout

Replace video=vesafb:off video=efifb:off with initcall_blacklist=sysfb_init as I suggested in my previous reply.

 

[Chapp]

yes sorry i sent the wrong line but anyway i still have the probleme :
here is the error inside windows : Windows a arrêté ce périphérique car il présente des problèmes. (Code 43)

 

[Chapp]

BOOT_IMAGE=/boot/vmlinuz-5.15.102-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt kvm.ignore_msrs=1 initcall_blacklist=sysfb_init

 

[leesteken]

Machine version 6.2 and initcall_blacklist=sysfb_init works for my RX570 on Windows 10. Maybe your GTX960 and/or the (older?) NVidia drivers needs additional work-arounds like dumping and/or patching the ROM-file. I don't know, sorry. Maybe you can find more specific advise in older threads on this forum or someone else might help you here.

 

[Chapp]

I don't think it's the GPU because i tried few days ago with my rtx2070s and same probleme.
thanks anyway

 

[Chapp]

i found something will doing dmesg | grep -i vfio i got those error :

[    3.448056] VFIO - User Level meta-driver version: 0.3
[    3.472432] vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem
[    3.491936] vfio_pci: add [10de:1401[ffffffff:ffffffff]] class 0x000000/00000000
[    3.515574] vfio_pci: add [10de:0fba[ffffffff:ffffffff]] class 0x000000/00000000
[  355.844094] vfio-pci 0000:01:00.0: vfio_ecap_init: hiding ecap 0x1e@0x258
[  355.844113] vfio-pci 0000:01:00.0: vfio_ecap_init: hiding ecap 0x19@0x900
[  355.845176] vfio-pci 0000:01:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff
[  355.863923] vfio-pci 0000:01:00.1: enabling device (0000 -> 0002)
[  361.327873] vfio-pci 0000:01:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff
[  492.389013] vfio-pci 0000:01:00.0: no suspend buffer for LTR; ASPM issues possible after resume
[  539.991419] vfio-pci 0000:01:00.0: no suspend buffer for LTR; ASPM issues possible after resume
[  576.739388] vfio-pci 0000:01:00.0: no suspend buffer for LTR; ASPM issues possible after resume
[  611.490441] vfio-pci 0000:01:00.0: no suspend buffer for LTR; ASPM issues possible after resume
[  770.987088] vfio-pci 0000:01:00.0: no suspend buffer for LTR; ASPM issues possible after resume
[  777.579008] vfio-pci 0000:01:00.0: no suspend buffer for LTR; ASPM issues possible after resume
[  779.837445] vfio-pci 0000:01:00.0: no suspend buffer for LTR; ASPM issues possible after resume
[  779.837494] vfio-pci 0000:01:00.0: no suspend buffer for LTR; ASPM issues possible after resume
[  779.837529] vfio-pci 0000:01:00.0: vfio_ecap_init: hiding ecap 0x1e@0x258
[  779.837539] vfio-pci 0000:01:00.0: vfio_ecap_init: hiding ecap 0x19@0x900
[  779.838442] vfio-pci 0000:01:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff
[  779.894999] vfio-pci 0000:01:00.0: no suspend buffer for LTR; ASPM issues possible after resume
[  783.268676] vfio-pci 0000:01:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff
[  818.009894] vfio-pci 0000:01:00.0: no suspend buffer for LTR; ASPM issues possible after resume
[  852.441021] vfio-pci 0000:01:00.0: no suspend buffer for LTR; ASPM issues possible after resume

 

[clavismil]

Hello, could you solve that issue? I'm having exactly the same problem with nvidia P400 Quadro.

I see similar line in dmesg

[  634.623306] vfio-pci 0000:01:00.0: vfio_ecap_init: hiding ecap 0x19@0x900
[  634.624565] vfio-pci 0000:01:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff
[  634.642930] vfio-pci 0000:01:00.1: enabling device (0000 -> 0002)
[  638.566293] vfio-pci 0000:01:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff

when trying to dump the .rom file i get

root@proxmox:~# cd /sys/bus/pci/devices/0000\:01\:00.0/
root@proxmox:/sys/bus/pci/devices/0000:01:00.0# echo 1 > rom
root@proxmox:/sys/bus/pci/devices/0000:01:00.0# cat rom > /tmp/image.rom
cat: rom: Input/output error

here I posted in r/Proxmox all my configs

 

[jjuk]

Adding "initcall_blacklist=sysfb_init" in Grub, plus changing Machine q35 version from 8.0 to 6.2 work for me. Thanks a lot

 

[8192K]

It's a bit crazy that this got it working... Could someone explain why or if there are any downsides to using "pc-q35-8.1", which I had before?

 

[leesteken]

It's a bit crazy that this got it working... Could someone explain why or if there are any downsides to using "pc-q35-8.1", which I had before?

The virtual machine (of type q35) are slightly different for each version. Windows and Windows drivers don't always handle certain virtual PCI(e) layout very well. That's why Proxmox fixates the version of a Windows VM when it is created but always uses the latest version on Linux VMs, and that's also why older machine versions sometimes work better (for older Windows).

 

[8192K]

But I'm using Windows 11. And the issue only appeared after passing a GPU after I already passed through 3 USB controllers. Only the controllers was no issue.

 

[leesteken]

But I'm using Windows 11. And the issue only appeared after passing a GPU after I already passed through 3 USB controllers. Only the controllers was no issue.

Then it's probably the Windows GPU driver that is picky about PCI(e) layout. I experienced this with AMD Radeon drivers a while back.