Windows 10: Nvidia GTX 1070 'Error 43'

B

brando56894

New Member
Dec 7, 2017
18
1
1
39
I know this is a long standing issue, and I know my GPU is able to be passed through because it works without a hitch in unRAID (I hate JBOD and prefer ZFS), and I've had it working in Arch as well. I just can't get it to work for the life of me!

Any help would be appreciated.

I've tried OMVF with PCI and PCI-E options, and also SeaBIOS with both PCI and PCI-E options, as per the wiki. OVMF with PCI-E seemed to give the most success since after installing Windows and installing GeForce experience it enables the soundcard and I can hear the notification tones as it installs all my USB devices. It installs the graphics driver, resets the screen, then it's almost entirely artifacted. If I reboot the VM it usually freezes during boot up, reboots and disables the driver and goes back to using the default driver and goes back to giving me error 43. The others don't even get to the point of enabling the audio.

Hardware
Intel Xeon E5-1650
Asus X99-WS/IPMI
64 GB DDR4 ECC
EVGA GTX 1070

VM Config:
Code: 
balloon: 8192
bootdisk: virtio0
cores: 4
cpu: host
efidisk0: VM_Storage:vm-103-disk-1,size=128K
hostpci0: 03:00,x-vga=on
hotplug: 0
ide0: iso:iso/virtio-win-0.1.141.iso,media=cdrom,size=309208K
ide2: iso:iso/Windows10.iso,media=cdrom,size=3633M
memory: 16384
name: HTPC-Windows
net0: virtio=0E:96:B2:5B:70:51,bridge=vmbr0,queues=4
numa: 0
ostype: win10
scsihw: virtio-scsi-single
smbios1: uuid=fc3389f5-4592-4a47-965b-f7ad1bbef899
sockets: 1
usb0: host=046d:c22b
usb1: host=046d:c22a
usb2: host=046d:c52b
usb3: host=1b1c:0c0a

IOMMU groups are properly separated

Code: 
IOMMU Group 29 03:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104 [GeForce GTX 1070] [10de:1b81] (rev a1)
IOMMU Group 29 03:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)

VFIO has control of the GPU

Code: 
 [root@proxmox ~] #> cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=10de:1b81,10de:10f0 disable_vga=1

 [root@proxmox ~] #> lspci -v
03:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1) (prog-if 00 [VGA controller])
        Subsystem: eVga.com. Corp. GP104 [GeForce GTX 1070]
        Physical Slot: 6
        Flags: fast devsel, IRQ 11, NUMA node 0
        Memory at f9000000 (32-bit, non-prefetchable) [disabled] [size=16M]
        Memory at c0000000 (64-bit, prefetchable) [disabled] [size=256M]
        Memory at d0000000 (64-bit, prefetchable) [disabled] [size=32M]
        I/O ports at d000 [disabled] [size=128]
        Expansion ROM at fa000000 [disabled] [size=512K]
        Capabilities: [60] Power Management version 3
        Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [78] Express Legacy Endpoint, MSI 00
        Capabilities: [100] Virtual Channel
        Capabilities: [250] Latency Tolerance Reporting
        Capabilities: [128] Power Budgeting <?>
        Capabilities: [420] Advanced Error Reporting
        Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
        Capabilities: [900] #19
        Kernel driver in use: vfio-pci
        Kernel modules: nvidiafb, nouveau

03:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1)
        Subsystem: eVga.com. Corp. GP104 High Definition Audio Controller
        Physical Slot: 6
        Flags: bus master, fast devsel, latency 0, IRQ 10, NUMA node 0
        Memory at fa080000 (32-bit, non-prefetchable) [size=16K]
        Capabilities: [60] Power Management version 3
        Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [78] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

I dumped the GPU ROM from within Windows while it was passed through and transferred it out, UEFI is supported, but when I try to use it with the VM I get no display. I couldn't dump it from within Linux because I was getting cat: rom: Input/output error

Code: 
 [root@proxmox rom-parser] #> ./rom-parser gp104.rom
Valid ROM signature found @a00h, PCIR offset 1a0h
        PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 1b81, class: 030000
        PCIR: revision 0, vendor revision: 1
Valid ROM signature found @fa00h, PCIR offset 1ch
        PCIR: type 3 (EFI), vendor: 10de, device: 1b81, class: 030000
        PCIR: revision 3, vendor revision: 0
                EFI: Signature Valid, Subsystem: Boot, Machine: X64
        Last image

msrs is ignored
Code: 
 [root@proxmox ~] #> cat /etc/modprobe.d/kvm.conf
options kvm ignore_msrs=1

full qemu command
Code: 
 [root@proxmox 0000:03:00.0] #> qm showcmd 103
/usr/bin/kvm -id 103 -chardev 'socket,id=qmp,path=/var/run/qemu-server/103.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -pidfile /var/run/qemu-server/103.pid -daemonize -smbios 'type=1,uuid=fc3389f5-4592-4a47-965b-f7ad1bbef899' -name HTPC-Windows -smp '4,sockets=1,cores=4,maxcpus=4' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga none -nographic -no-hpet -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=proxmox,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,kvm=off' -m 16384 -k en-us -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pci.0,addr=0x5' -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -readconfig /usr/share/qemu-server/pve-usb.cfg -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'vfio-pci,host=03:00.0,id=hostpci0.0,bus=pci.0,addr=0x10.0,x-vga=on,multifunction=on' -device 'vfio-pci,host=03:00.1,id=hostpci0.1,bus=pci.0,addr=0x10.1' -device 'usb-host,vendorid=0x046d,productid=0xc22b,id=usb0' -device 'usb-host,vendorid=0x046d,productid=0xc22a,id=usb1' -device 'usb-host,vendorid=0x046d,productid=0xc52b,id=usb2' -device 'usb-host,vendorid=0x1b1c,productid=0x0c0a,id=usb3' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:359df93fc897' -drive 'file=/mnt/storage/stuff/iso/template/iso/virtio-win-0.1.141.iso,if=none,id=drive-ide0,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.0,unit=0,drive=drive-ide0,id=ide0,bootindex=200' -drive 'file=/mnt/storage/stuff/iso/template/iso/Windows10.iso,if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=201' -drive 'file=/dev/zvol/nvme/vms/vm-103-disk-2,if=none,id=drive-virtio0,format=raw,cache=none,aio=native,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap103i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on,queues=4' -device 'virtio-net-pci,mac=0E:96:B2:5B:70:51,netdev=net0,bus=pci.0,addr=0x12,id=net0,vectors=10,mq=on,bootindex=300' -rtc 'driftfix=slew,base=localtime' -global 'kvm-pit.lost_tick_policy=discard'
 
Last edited:
Code: 
 [root@proxmox ~] #> pveversion -v
proxmox-ve: 5.1-30 (running kernel: 4.13.8-3-pve)
pve-manager: 5.1-38 (running version: 5.1-38/1e9bc777)
pve-kernel-4.13.4-1-pve: 4.13.4-26
pve-kernel-4.13.8-2-pve: 4.13.8-28
pve-kernel-4.13.8-3-pve: 4.13.8-30
libpve-http-server-perl: 2.0-7
lvm2: 2.02.168-pve6
corosync: 2.4.2-pve3
libqb0: 1.0.1-1
pve-cluster: 5.0-19
qemu-server: 5.0-17
pve-firmware: 2.0-3
libpve-common-perl: 5.0-22
libpve-guest-common-perl: 2.0-13
libpve-access-control: 5.0-7
libpve-storage-perl: 5.0-17
pve-libspice-server1: 0.12.8-3
vncterm: 1.5-3
pve-docs: 5.1-12
pve-qemu-kvm: 2.9.1-3
pve-container: 2.0-17
pve-firewall: 3.0-5
pve-ha-manager: 2.0-4
ksm-control-daemon: 1.2-2
glusterfs-client: 3.8.8-1
lxc-pve: 2.1.1-2
lxcfs: 2.0.8-1
criu: 2.11.1-1~bpo90
novnc-pve: 0.6-4
smartmontools: 6.5+svn4324-1
zfsutils-linux: 0.7.3-pve1~bpo9
 
mhmm i have a setup here that works, (with ovmf and pcie)
but some things are (probably different)
an i7 cpu (not xeon)
i have a dedicated gpu for the host (your post does not say if you do)
i have not set dynamic memory allocation
and my disk is virtio-scsi instead of virtio-blk
 
I have an ASM2400 integrated VGA adapter, which isn't in use.

Code: 
09:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 30)

My storage is backed by ZFS, but the virtio-blk instead of scsi was a mistake. I've installed Windows and recreated the VM a few different times, I was originally using scsi.
 
brando56894 said:
I have an ASM2400 integrated VGA adapter, which isn't in use.
Click to expand...
can you choose in the mainboard which vga adapter to boot off ? if yes try to boot via the ASM2400
you could also try to change the pcie slot of the gpu (if you have more than one)

nothing else comes to mind..
 
Yep, I have the internal adapter set as the primary. I currently have it in the third PCI-E slot since that's what the motherboard recommends.
 
Hmm, so it does. I could have sworn that the Firmware told me slot 3 was the optimal one. I'll switch it and give it a try.
 
Hi Brando!

You didn't set the machine entry in the kvm config
Code: 
machine: q35
for pcie to work properly.
 
Hey Oliver, I've tried it multiple ways, so it wasn't included above, but it give the most promising results.

@dcsapak I switched it over to slot 2 and passed it through to a newly created VM with OVMF and PCI passthrough....and I'm still getting error 43 :(

Code: 
 [root@proxmox ~] #> cat /etc/pve/qemu-server/103.conf
balloon: 0
bios: ovmf
bootdisk: scsi0
cores: 4
cpu: host
efidisk0: VM_Storage:vm-103-disk-2,size=128K
hostpci0: 01:00,x-vga=on
ide0: iso:iso/virtio-win-0.1.141.iso,media=cdrom,size=309208K
ide2: iso:iso/Windows10.iso,media=cdrom
memory: 16384
name: HTPC-Windows
net0: virtio=FE:1C:3D:3D:97:03,bridge=vmbr0,queues=4
numa: 0
ostype: win10
scsi0: VM_Storage:vm-103-disk-1,size=64G
scsihw: virtio-scsi-pci
smbios1: uuid=317aa668-0c42-4d30-9d2b-855e3ecb8c00
sockets: 1
usb0: host=046d:c22b
usb1: host=046d:c22a
usb2: host=046d:c52b
usb3: host=1b1c:0c0a
 
you could try to deactivate the hyper-v enlightenments:
add to your config:

Code: 
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=proxmox,kvm=off'

you can also try to set the hv_vendor_id=proxmox to something else, but i doubt nvidia looks for the 'proxmox' string
 
When I tried to turn KVM off before and set my CPU as host it told me it required KVM.

Well good news is it's now working correctly! What a pain in the a$$! After the last attempt posted above, I shut down, added machine: q35 and added pcie=1, booted it up and let Windows reinstall the GPU drivers and she worked! I rebooted just to make sure it would stick and it did.

Does it matter if pcie=1 is before x-vga=on? I believe it was always after and just to be meticulous this time, I put it before and now it works. Hopefully it stays that way.
 
Unmarking this as 'solved' because the way that solves 'error 43' gives me horrible audio and video quality.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom