Is anybody who succeeded GPU passthrough (NVIDIA) with Windows 7 ?

Stargazer

New Member
Jan 25, 2018
6
0
1
32
Thank you for enduring my bad English, first. ^ _^ ;

I used KVM with libvirt (virt-manager) and 'migrated' to Proxmox recently.

I had succeeded GPU passthrough in libvirt with win7 win8.1 win10 but now I just do it with win10 with Proxmox.
(win 10 seems very easy as it requires 'almost' only GPU rom)

Win 8.1 seems to be easy but it isn't because it looks succeeded but the screen display isn't
although it shows menu "Nvidia Control Panel" and 3d accelated Chrome doesn't show up (looks white blank screen.

Win 7 is stuck on boot ( I suppose with cpu usage because I can't look at without VNC)
My settings is


....
machine: q35
hostpci0: 01:00.0;01:00.1, pcie=1, x-vga=on, romfile=1.rom
.....

- pre-required works are done (iommu, vfio, acs patch, rom file ......)
- installed Win 7 as UEFI with OVMF
- boot option edited with boot up screen and set only "PCI-E Device" boot
- manually updated kernel and qemu-kvm
- added efi disk

I set almost similar circumtances with similar options as libvirt with virt-manager,
but just stuck on boot in Proxmox
Is there any hints?

Thanks in advance.
 
Last edited:

frickelpit

New Member
Oct 29, 2017
15
0
1
47
I have the same problem. Win7 stucks on the bootscreen logo. With RDP connect you can see, the grafic card becomes a code 12 error in the device manager. No matter if UEFI or seabios. The result is the same. I think, it's actually not possible because nobody has a running config with gpu passthrough and proxmox 5.2.
 
Last edited:

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
3,931
359
83
31
Vienna
mhmm, could you post the output of 'qm showcmd ID' ?
maybe some cpu/hv flags are not good for windows 7
 

frickelpit

New Member
Oct 29, 2017
15
0
1
47
No problem...

Code:
/usr/bin/kvm -id 205 -chardev 'socket,id=qmp,path=/var/run/qemu-server/205.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -pidfile /var/run/qemu-server/205.pid -daemonize -smbios 'type=1,uuid=d4c04550-2da2-447e-8d4c-a2b38f192ed0' -drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/kvm/OVMF_CODE-pure-efi.fd' -drive 'if=pflash,unit=1,format=raw,id=drive-efidisk0,file=/dev/pve/vm-205-disk-2' -name WS5 -smp '2,sockets=1,cores=2,maxcpus=2' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga none -nographic -cpu 'kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce,kvm=off' -m 4096 -k de -object 'iothread,id=iothread-virtio1' -readconfig /usr/share/qemu-server/pve-q35.cfg -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=01:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on' -device 'vfio-pci,host=01:00.1,id=hostpci0.1,bus=ich9-pcie-port-1,addr=0x0.1' -device 'usb-host,vendorid=0x05ac,productid=0x0221,id=usb0' -device 'usb-host,vendorid=0x045e,productid=0x00cb,id=usb1' -device 'usb-host,vendorid=0x04b8,productid=0x011f,id=usb2' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:17d28020a41f' -drive 'file=/pool1/TEMPLATES/template/iso/virtio-win-0.1.141.iso,if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2' -drive 'file=/dev/pve/vm-205-disk-1,if=none,id=drive-virtio0,cache=writeback,format=raw,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -drive 'file=/dev/zvol/pool1/vm-205-disk-1,if=none,id=drive-virtio1,cache=writeback,format=raw,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio1,id=virtio1,bus=pci.0,addr=0xb,iothread=iothread-virtio1' -netdev 'type=tap,id=net0,ifname=tap205i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=B2:6E:2F:28:3D:8A,netdev=net0,bus=pci.0,addr=0x12,id=net0' -machine 'type=q35'
 

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
3,931
359
83
31
Vienna
can you try to set the windows version to >= win8 (even though its running win7)?
 

Stargazer

New Member
Jan 25, 2018
6
0
1
32
In libvirt/virt-manager, Win7 was successful because I can directly assign PCI-E device to boot device.

I finally know it's unable in Proxmox because I can't assign GPU to boot.

Windows 8.1, as I said above, it seems work normally but it doesn't show 3D part of screen.
(as I know till now)
 
Last edited:

JustAMan

New Member
Jul 31, 2018
8
3
3
30
Hello there!

I've found the relevant bug against qemu-kvm,. google for "Red Hat Bugzilla – Bug 1273172" (I cannot post direct link to this bug as I'm a new user and not allowed to post external links so far).

Following the ideas in that bug I was able to bring up a Win7 VM with NVidia GTX 1080 Ti passed through (last night Unigine Heaven on 4K output with everything on max yielded a score of about 970, while I see native results ranging from 980 to 1010, so not that bad).

The problem haunting my configuration (error code 12 - not enough resources) has something to do with the fact that by default host PCI devices are passed through as though they are behind a PCI bridge (which works just fine on Win10 and MacOS High Sierra, BTW, on the same hardware).

Below is the configuration file for my VM. I have passed through VGA, HDMI audio and a USB controller.

Code:
vga: none
agent: 1
args: -cpu host,hv_time,kvm=off,hv_vendor_id=TheNeverhood,-hypervisor -device vfio-pci,host=02:00.0,id=hostpci0.0,bus=pcie.0,addr=0x10.0,multifunction=on -device vfio-pci,host=02:00.1,id=hostpci0.1,bus=pcie.0,addr=0x10.1
balloon: 0
bios: ovmf
boot: dcn
bootdisk: scsi1
cores: 4
cpu: host
efidisk0: local:101/vm-101-disk-2.qcow2,size=128K
hostpci1: 00:14.0,pcie=1
ide2: none,media=cdrom
machine: q35
memory: 8192
name: win7main
net0: virtio=E2:68:03:36:43:6A,bridge=vmbr0
numa: 0
ostype: win7
scsi1: local:101/vm-101-disk-1.qcow2,discard=on,iothread=1,size=120G
scsihw: virtio-scsi-pci
smbios1: uuid=4c2c2891-99b8-4287-b057-9f2aeaad7692
sockets: 1
Hope that helps other people interested in such a thing as Windows 7 with passed through video card... :)
 

frickelpit

New Member
Oct 29, 2017
15
0
1
47
Best hint ever! I have tried so many hours to run this shitty W7 with passed through video card. I always run into error 12. Now i can use my old CAD soft virtualised. GREAT!!!
 

frickelpit

New Member
Oct 29, 2017
15
0
1
47
After testing a while i have to say unfortunately it works only one time. After shutting down the win7 vm, it is not possible to start again or to start another vm for example win10 with gpu passthrough. The host system freeze shortly after vm start :(

I think that the proxmox mechanism to release the grafic card isn't made for that use case.... ?
 

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
3,931
359
83
31
Vienna
I think that the proxmox mechanism to release the grafic card isn't made for that use case.... ?
unfortunately this is more related to the hardware and the guest os (which is responsible to release the devices)
 

JustAMan

New Member
Jul 31, 2018
8
3
3
30
My own setup (ASUS Z170 PRO motherboard + NVidia 1080 Ti video) works fine so far.
I did not give it any significant load, though, just ran Unigine Heaven a couple of times.

What video card do you have?
 

frickelpit

New Member
Oct 29, 2017
15
0
1
47
I use a Fujitsu DS D3417-B2 Mobo with a XEON E3-1245 v6 and 64GB ECC-RAM. Grafic card is a NVidia Quadro P2000 with driver v391.89 WHQL for Win7x64.
 

JustAMan

New Member
Jul 31, 2018
8
3
3
30
I have tinkered my own version of qemu-server that is able to control how to pass the card (as it seems it does a few more things when you put "x-vga=on" in the hostpci line compared to my initial config I posted above).
Let me get back to home and I can try and show you qemu command line so we can try to compare mine and yours and see if there's some difference.

Then you can try to construct a commandline yourself and see if patched commandline fixes your issues.
 

JustAMan

New Member
Jul 31, 2018
8
3
3
30
Here is the result of qm showcmd 101 (101 is the id of my Win7 VM):

/usr/bin/kvm -id 101 -name win7main -chardev 'socket,id=qmp,path=/var/run/qemu-server/101.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -pidfile /var/run/qemu-server/101.pid -daemonize -smbios 'type=1,uuid=4c2c2891-99b8-4287-b057-9f2aeaad7692' -drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' -drive 'if=pflash,unit=1,format=qcow2,id=drive-efidisk0,file=/var/lib/vz/images/101/vm-101-disk-2.qcow2' -smp '6,sockets=1,cores=6,maxcpus=6' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga none -nographic -no-hpet -cpu 'Skylake-Client,+kvm_pv_unhalt,+kvm_pv_eoi,enforce,kvm=off,vendor=GenuineIntel' -m 8192 -object 'iothread,id=iothread-virtioscsi1' -readconfig /usr/share/qemu-server/pve-q35.cfg -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=02:00.0,id=hostpci0.0,bus=pcie.0,addr=0x10.0,multifunction=on' -device 'vfio-pci,host=02:00.1,id=hostpci0.1,bus=pcie.0,addr=0x10.1' -device 'vfio-pci,host=00:14.0,id=hostpci1,bus=ich9-pcie-port-2,addr=0x0' -chardev 'socket,path=/var/run/qemu-server/101.qga,server,nowait,id=qga0' -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:cd1a3fd61c95' -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -device 'virtio-scsi-pci,id=virtioscsi1,bus=pci.3,addr=0x2,iothread=iothread-virtioscsi1' -drive 'file=/var/lib/vz/images/101/vm-101-disk-1.qcow2,if=none,id=drive-scsi1,discard=on,format=qcow2,cache=none,aio=native,detect-zeroes=unmap' -device 'scsi-hd,bus=virtioscsi1.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1,id=scsi1,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap101i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' -device 'e1000,mac=E2:68:03:36:43:6A,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -rtc 'driftfix=slew,base=localtime' -machine 'type=q35' -global 'kvm-pit.lost_tick_policy=discard'

Try comparing with yours to see if there's any difference.
 
  • Like
Reactions: frickelpit

frickelpit

New Member
Oct 29, 2017
15
0
1
47
Thx. I have tested it, but nothing helps. Now my vm.cfg looks like yours. But with the difference that my second card which i passthrough to the vm is a pci (USB3-Card) and not a pcie card. I have tried to add the device-line from the USB3-Card to the args-line in cfg, but Proxmox find this not usefull :(

If i run this command at console proxmox can't find "bus=pci.0", if i start same cfg with webgui the vm starts without a problem?!
Code:
/usr/bin/kvm -id 203 -name CAD7 -chardev 'socket,id=qmp,path=/var/run/qemu-server/203.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -pidfile /var/run/qemu-server/203.pid -daemonize -smbios 'type=1,uuid=a7d9be4d-014c-476f-b42f-ab7bc1738471' -drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' -drive 'if=pflash,unit=1,format=raw,id=drive-efidisk0,file=/dev/pve/vm-203-disk-2' -smp '8,sockets=1,cores=8,maxcpus=8' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga none -nographic -no-hpet -cpu 'Skylake-Client,+kvm_pv_unhalt,+kvm_pv_eoi,enforce,kvm=off,vendor=GenuineIntel' -m 8192 -object 'iothread,id=iothread-virtio1' -readconfig /usr/share/qemu-server/pve-q35.cfg  -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=01:00.0,id=hostpci0.0,bus=pcie.0,addr=0x10.0,multifunction=on' -device 'vfio-pci,host=01:00.1,id=hostpci0.1,bus=pcie.0,addr=0x10.1' -device 'vfio-pci,host=02:00.0,id=hostpci1,bus=pci.0,addr=0x11' -chardev 'socket,path=/var/run/qemu-server/203.qga,server,nowait,id=qga0' -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:17d28020a41f' -drive 'file=/dev/pve/vm-203-disk-1,if=none,id=drive-virtio0,cache=writeback,format=raw,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -drive 'file=/dev/zvol/pool1/vm-203-disk-1,if=none,id=drive-virtio1,cache=writeback,format=raw,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio1,id=virtio1,bus=pci.0,addr=0xb,iothread=iothread-virtio1' -netdev 'type=tap,id=net0,ifname=tap203i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=B2:6E:2F:28:3D:8A,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -rtc 'driftfix=slew,base=localtime' -machine 'type=q35' -global 'kvm-pit.lost_tick_policy=discard'

btw... the device manager in win7 is free of errors or warnings. The problem comes after the shutdown from win7. Start another vm and the host will be frozen.
 
Last edited:

JustAMan

New Member
Jul 31, 2018
8
3
3
30
What happens if you start this VM again after shutting it down, does it work? I can not test other VMs now as I don't have them yet.
Also, what is the output when you double-click "Start" task in task log? And is there anything interesting in dmesg after turning problematic VM on or off?
 

frickelpit

New Member
Oct 29, 2017
15
0
1
47
Problem solved!
The reason was the USB3 PCI-Card which i want to passthrough to the Win-VMs. At host start the card becomes the kernel driver "xhci_hcd", same as the host use for his USB3-chip. WIN10 has no problems with this, differently to WIN7. I haven't found a way to prevent the card from loading the driver because it's a kernel driver and not a modul.

So i passthrough the usb ports and not the hole pci card. Not nice but it works :)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!