[SOLVED] Error 43 on Win11 with MSI GT 1030 passthrough

M

MrKings

New Member
Jul 25, 2023
10
0
1
Hi. I'm trying to passthrough an MSI GT 1030 2G LP OC to a Windows 11 VM. I've managed to boot, install drivers, and for a brief second after installing NVIDIA drivers I get the device working without issues, only to then get error 43. I've tried so much different configurations to try to fix this, that it is a bit difficult to list everything I tried.
I'm running Proxmox on a machine with:
  • AMD Ryzen 7950X
  • ASRock B650 PG Lightning
  • 2x MSI GT 1030 2G LP OC
  • UEFI enable / CSM disabled
  • Resizable BAR disabled
  • SVM / SR-IOV / IOMMU enabled
  • AMD iGPU set as the primary monitor on BIOS and dummy HDMI plugged in to MB
Tried to passthrough to a Ubuntu machine and it works as expected.

As of now, I have the following config:

Code: 
cat /etc/pve/qemu-server/100.conf
agent: 1
args: -cpu 'host,kvm=off,hv_vendor_id=null' -machine 'kernel_irqchip=on'
balloon: 10240
bios: ovmf
boot: order=scsi0;net0
cores: 32
cpu: host,hidden=1
cpulimit: 28
efidisk0: local-lvm:vm-100-disk-0,efitype=4m,pre-enrolled-keys=1,size=4M
hostpci0: 0000:01:00,pcie=1,romfile=GP108_patched.rom,x-vga=1
kvm: 1
machine: pc-q35-8.0
memory: 16384
meta: creation-qemu=7.2.0,ctime=1687185593
name: joao-dev-machine
net0: virtio=7A:4B:32:2C:06:91,bridge=vmbr0,firewall=1
numa: 0
ostype: win11
scsi0: local-lvm:vm-100-disk-1,iothread=1,size=384G
scsihw: virtio-scsi-single
smbios1: uuid=8a52e561-d601-4b25-870b-eef58b779c8a
sockets: 1
tpmstate0: local-lvm:vm-100-disk-2,size=4M,version=v2.0
usb0: host=1908:0226
usb1: host=05ac:024f
usb2: host=046d:082d
usb3: host=047d:00f2
usb4: host=046d:c548
vga: none
vmgenid: cb04cfc0-4bdb-485e-a3fe-57c86fe7098a
VBIOS was dumped and patched by myself

Code: 
pveversion -v
proxmox-ve: 8.0.1 (running kernel: 6.2.16-4-pve)
pve-manager: 8.0.3 (running version: 8.0.3/bbf3993334bfa916)

Code: 
cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-6.2.16-4-pve root=/dev/mapper/pve-root ro quiet iommu=pt initcall_blacklist=sysfb_init kvm_amd.npt=1 kvm_amd.avic=1 kvm_amd.nested=1 kvm_amd.sev=1 vfio-pci.ids=10de:1d01,10de:0fb8 vfio_iommu_type1.allow_unsafe_interrupts=1 kvm.ignore_msrs=1 kvm.report_ignored_msrs=0 nouveau.modeset=0 modprobe.blacklist=nouveau,nvidia,nvidiafb,nvidia-gpu

Code: 
lsmod | grep vfio
vfio_pci               16384  0
vfio_pci_core          94208  1 vfio_pci
irqbypass              16384  75 vfio_pci_core,kvm
vfio_iommu_type1       49152  0
vfio                   57344  3 vfio_pci_core,vfio_iommu_type1,vfio_pci
iommufd                73728  1 vfio

Code: 
IOMMU Group 0:
    00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14da]
IOMMU Group 1:
    00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:14db]
IOMMU Group 2:
    00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14da]
IOMMU Group 3:
    00:02.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:14db]
IOMMU Group 4:
    00:02.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:14db]
IOMMU Group 5:
    00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14da]
IOMMU Group 6:
    00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14da]
IOMMU Group 7:
    00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14da]
IOMMU Group 8:
    00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:14dd]
IOMMU Group 9:
    00:08.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:14dd]
IOMMU Group 10:
    00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 71)
    00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU Group 11:
    00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e0]
    00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e1]
    00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e2]
    00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e3]
    00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e4]
    00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e5]
    00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e6]
    00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e7]
IOMMU Group 12:
    01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP108 [GeForce GT 1030] [10de:1d01] (rev a1)
    01:00.1 Audio device [0403]: NVIDIA Corporation GP108 High Definition Audio Controller [10de:0fb8] (rev a1)
IOMMU Group 13:
    02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f4] (rev 01)
IOMMU Group 14:
    03:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
    04:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP108 [GeForce GT 1030] [10de:1d01] (rev a1)
    04:00.1 Audio device [0403]: NVIDIA Corporation GP108 High Definition Audio Controller [10de:0fb8] (rev a1)
IOMMU Group 15:
    03:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 16:
    03:03.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 17:
    03:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 18:
    03:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 19:
    03:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 20:
    03:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 21:
    03:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 22:
    03:0a.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 23:
    03:0b.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
    0d:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU Group 24:
    03:0c.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
    0e:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f7] (rev 01)
IOMMU Group 25:
    03:0d.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
    0f:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f6] (rev 01)
IOMMU Group 26:
    10:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd Device [144d:a80c]
IOMMU Group 27:
    11:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Raphael [1002:164e] (rev c1)
IOMMU Group 28:
    11:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Rembrandt Radeon High Definition Audio Controller [1002:1640]
IOMMU Group 29:
    11:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] VanGogh PSP/CCP [1022:1649]
IOMMU Group 30:
    11:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:15b6]
IOMMU Group 31:
    11:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:15b7]
IOMMU Group 32:
    11:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h/19h HD Audio Controller [1022:15e3]
IOMMU Group 33:
    12:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:15b8]

If anyone could help me get through error 43 I would be very grateful.
 
Last edited:
Same exact issue on Windows 11. Works briefly then reverts. I have been trying different kernels and am trying to use 5.3 as I've heard that is most stable.
 
What's the output of lspci -k | grep -A 2 -i '01:00'
Which nvidia driver version are you using in windows?
 
jagson said:
What's the output of lspci -k | grep -A 2 -i '01:00'
Which nvidia driver version are you using in windows?
Click to expand...
Code: 
lspci -k | grep -A 2 -i '01:00'
01:00.0 VGA compatible controller: NVIDIA Corporation GP108 [GeForce GT 1030] (rev a1)
    Subsystem: Micro-Star International Co., Ltd. [MSI] GP108 [GeForce GT 1030]
    Kernel driver in use: vfio-pci
--
01:00.1 Audio device: NVIDIA Corporation GP108 High Definition Audio Controller (rev a1)
    Subsystem: Micro-Star International Co., Ltd. [MSI] GP108 High Definition Audio Controller
    Kernel driver in use: vfio-pci

I'm using the latest driver version at this moment, 536.67.
 
I started to have issues with my GV-N1030D5-2GL too, although it was working with the same settings couple month back.
Plugged it back today, worked ok only until selected as primary GPU ( x-vga=1 ).
Meanwhile my GV-N1030D4-2GL (GDDR4 version) worked as expected with the same settings except different vBIOS rom.

Solved by dumping vBIOS and patching again.
I still had original dump from month back, and it does not even match in size, so they must have auto-flashed it via driver update or something..
 
InQuize said:
I started to have issues with my GV-N1030D5-2GL too, although it was working with the same settings couple month back.
Plugged it back today, worked ok only until selected as primary GPU ( x-vga=1 ).
Meanwhile my GV-N1030D4-2GL (GDDR4 version) worked as expected with the same settings except different vBIOS rom.

Solved by dumping vBIOS and patching again.
I still had original dump from month back, and it does not even match in size, so they must have auto-flashed it via driver update or something..
Click to expand...
So this is still a result of Nvidia not wanting their cards to run in VMs?
 
What troubles me is that the card runs well on a Linux VM... This might indicate that it's the Windows driver not wanting to work...
 
InQuize said:
I started to have issues with my GV-N1030D5-2GL too, although it was working with the same settings couple month back.
Plugged it back today, worked ok only until selected as primary GPU ( x-vga=1 ).
Meanwhile my GV-N1030D4-2GL (GDDR4 version) worked as expected with the same settings except different vBIOS rom.

Solved by dumping vBIOS and patching again.
I still had original dump from month back, and it does not even match in size, so they must have auto-flashed it via driver update or something..
Click to expand...
I have Windows 11 on a USB drive, I will try to install the latest NVIDIA driver and dump the VBIOS again, and see if that changes anything.
 
MrKings said:
What troubles me is that the card runs well on a Linux VM... This might indicate that it's the Windows driver not wanting to work...
Click to expand...
I do not have desktop linux install handy to test, but it might not require romfile at all.
The patch we are applying is Win code 43 specific indeed, isn't it?

williamions said:
So this is still a result of Nvidia not wanting their cards to run in VMs?
Click to expand...
vBios is an OEM specific firmware, so may be has to do with Gigabyte, if other 1030s were not affected.
 
InQuize said:
I do not have desktop linux install handy to test, but it might not require romfile at all.
The patch we are applying is Win code 43 specific indeed, isn't it?


vBios is an OEM specific firmware, so may be has to do with Gigabyte, if other 1030s were not affected.
Click to expand...
Any updates on this? Still haven't been able to get it to work. Thinking maybe we need an older kernel and driver combo?
 
williamions said:
Any updates on this? Still haven't been able to get it to work. Thinking maybe we need an older kernel and driver combo?
Click to expand...
No. Issue is somewhere in your dumping or patching process.

I found this Win GUI wrapper ( RecreationalGarbage / NVIDIA-vBIOS-VFIO-Patcher-GUI ) to be easy and reliable for patching.
And used nvflash CLI tool (Linux) version to dump vBios from PVE host, although Win version of nvflash is easier to find.
Code: 
./nvflash --list
./nvflash --index=0 --save rom.bin
To transfer binary between platforms used:
Code: 
curl --upload-file 'rom.bin' 'https://paste.c-net.org/'
along with wget of patched rom from Nextcloud.
 
InQuize said:
No. Issue is somewhere in your dumping or patching process.

I found this Win GUI wrapper ( RecreationalGarbage / NVIDIA-vBIOS-VFIO-Patcher-GUI ) to be easy and reliable for patching.
And used nvflash CLI tool (Linux) version to dump vBios from PVE host, although Win version of nvflash is easier to find.
Code: 
./nvflash --list
./nvflash --index=0 --save rom.bin
To transfer binary between platforms used:
Code: 
curl --upload-file 'rom.bin' 'https://paste.c-net.org/'
along with wget of patched rom from Nextcloud.
Click to expand...
Sorry, I don't understand the methodology: I've dumped the vBios already with CPU-Z on Windows. Should I now copy that file to an Ubuntu VM, run the above script, and somehow upload it to Nextcloud for some reason?


I guess more broadly, didn't Nvidia decide to allow GPU passthrough obviating the need for patching? Not clear to me why this is still necessary.
 
williamions said:
didn't Nvidia decide to allow GPU passthrough obviating the need for patching? Not clear to me why this is still necessary.
Click to expand...
Don't know where those 'news' about nvidia making anything easier at all are coming from, but it is definitely necessary to patch nowadays.
In fact, it was not a thing before (pre PVE 7 I believe), so passthroug became more complicated if anything.

williamions said:
I've dumped the vBios already with CPU-Z on Windows.
Click to expand...
If it was baremetal and not a VM, then it may be sufficient. Though CPU-Z did fail me at least once in this process IIRC, that's why I mention nvflash, it is a more reliable tool for the job.
Sure, try your CPU-Z dump, but YMMV.
But make sure it is a recent dump, as Geforce firmwares do appear to auto-update silently.

williamions said:
Should I now copy that file to an Ubuntu VM, run the above script, and somehow upload it to Nextcloud for some reason?
Click to expand...
Just use GUI patcher I linked above and copy resulting rom to /usr/share/kvm of your Promox host by any means (I simply mentioned what was used in my case).
 
Last edited:
MrKings said:
I have Windows 11 on a USB drive, I will try to install the latest NVIDIA driver and dump the VBIOS again, and see if that changes anything.
Click to expand...
I installed the latest driver and dumped VBIOS again, which has no difference from the one I dumped before.

InQuize said:
No. Issue is somewhere in your dumping or patching process.

I found this Win GUI wrapper ( RecreationalGarbage / NVIDIA-vBIOS-VFIO-Patcher-GUI ) to be easy and reliable for patching.
And used nvflash CLI tool (Linux) version to dump vBios from PVE host, although Win version of nvflash is easier to find.
Code: 
./nvflash --list
./nvflash --index=0 --save rom.bin
To transfer binary between platforms used:
Code: 
curl --upload-file 'rom.bin' 'https://paste.c-net.org/'
along with wget of patched rom from Nextcloud.
Click to expand...

For patching, I'm not using this GUI but I'm using the underlying script.
Which kernel version are you using?
 
InQuize said:
Don't know where those 'news' about nvidia making anything easier at all are coming from, but it is definitely necessary to patch nowadays.
In fact, it was not a thing before (pre PVE 7 I believe), so passthroug became more complicated if anything.
Click to expand...
Almost two years ago NVIDIA released this statement.
 
MrKings said:
I installed the latest driver and dumped VBIOS again, which has no difference from the one I dumped before.
Click to expand...
Ok, so this was Gigabyte fixing something in my case then.

MrKings said:
Which kernel version are you using?
Click to expand...
Code: 
proxmox-ve: 7.4-1 (running kernel: 5.15.108-1-pve)
pve-manager: 7.4-16 (running version: 7.4-16/0f39f621)
I'll find some time to upgrade PVE and see if it breaks.
 
MrKings said:
I installed the latest driver and dumped VBIOS again, which has no difference from the one I dumped before.



For patching, I'm not using this GUI but I'm using the underlying script.
Which kernel version are you using?
Click to expand...
Were you able to get it to run?
 
Upgrade to v8 went smooth.

pve8.jpg

Code: 
root@office-pve:~# cat /etc/pve/qemu-server/101.conf

agent: 1
balloon: 0
bios: ovmf
boot: order=virtio0
cores: 8
cpu: host,flags=+md-clear;+pcid;+spec-ctrl;+ssbd;+aes
efidisk0: local-zfs:vm-101-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:01:00,pcie=1,romfile=GT1030_GV-N1030D5-2GL_fix.rom,x-vga=1
hostpci1: 0000:06:00,pcie=1,romfile=GT1030_GV-N1030D4-2GL_fix.rom
hotplug: network,usb
ide2: local:iso/kvm-iso.iso,media=cdrom,size=372416K
machine: pc-q35-8.0
memory: 10240
meta: creation-qemu=7.2.0,ctime=1679233630
name: win10
net0: virtio=96:51:6B:80:DE:9D,bridge=vmbr0,queues=8
numa: 0
onboot: 1
ostype: win10
parent: SketchUp
scsihw: virtio-scsi-single
smbios1: uuid=
sockets: 1
tablet: 1
tags: 
tpmstate0: local-zfs:vm-101-disk-1,size=4M,version=v2.0
usb0: host=1-9
usb1: host=1-5
vga: none
virtio0: local-zfs:vm-101-disk-2,backup=0,cache=writeback,discard=on,iothread=1,size=64G
vmgenid:

Code: 
root@office-pve:~# cat /etc/kernel/cmdline

root=ZFS=rpool/ROOT/pve-1 boot=zfs gfxpayload=text intel_iommu=on vfio-pci.disable_vga=1 iommu=pt pcie_acs_override=downstream,multifunction kvm.ignore_msrs=1 kvm.report_ignored_msrs=0 vfio-pci.ids=8086:191d,10de:1288,10de:1d01
# pci=assign-busses pci=realloc
# nofb nomodeset video=vesafb:off,efifb:off,simplefb:off
# i915.enable_gvt=1 i915.enable_dc=0
# i915.enable_guc=0
# i915.enable_dc=0 intel_idle.max_cstate=1 ahci.mobile_lpm_policy=1
 
Last edited:
  • Like
Reactions: _gabriel
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom