Ubuntu + PCIE GPU Pass Thru - TASK ERROR: Failed to run vncproxy.

[jeffs42885]

I just got my proxmox server setup using 2 pci-e devices.

Quadro K2200 GPU
HBA card

the HBA card passes through to a VM image of TrueNas - But I cannot get the GPU pass through for ubuntu to work at all. I followed the steps in this link:

https://www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/

When I go to install ubuntu - I get the following errors. Keep in mind, when i remove the PCI device (GPU) install works fine.

TASK ERROR: start failed: command '/usr/bin/kvm -id 101 -name 'ubuntu,debug-threads=on' -no-shutdown -chardev 'socket,id=qmp,path=/var/run/qemu-server/101.qmp,server=on,wait=off' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/101.pid -daemonize -smbios 'type=1,uuid=508eecf2-bb00-46e6-b2c7-126ece8635b8' -drive 'if=pflash,unit=0,format=raw,readonly=on,file=/usr/share/pve-edk2-firmware//OVMF_CODE_4M.secboot.fd' -drive 'if=pflash,unit=1,id=drive-efidisk0,format=raw,file=/dev/pve/vm-101-disk-2,size=540672' -smp '4,sockets=1,cores=4,maxcpus=4' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vnc 'unix:/var/run/qemu-server/101.vnc,password=on' -cpu 'host,kvm=off,+kvm_pv_eoi,+kvm_pv_unhalt,+pcid' -m 8192 -object 'iothread,id=iothread-virtioscsi0' -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg -device 'vmgenid,guid=60191b25-74e0-4b5e-b298-63ddfcecb23c' -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=0000:03:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on' -device 'vfio-pci,host=0000:03:00.1,id=hostpci0.1,bus=ich9-pcie-port-1,addr=0x0.1' -device 'VGA,id=vga,bus=pcie.0,addr=0x1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:cb2cca226c57' -drive 'file=/var/lib/vz/template/iso/ubuntu-22.04.1-live-server-amd64.iso,if=none,id=drive-ide2,media=cdrom,aio=io_uring' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=100' -device 'virtio-scsi-pci,id=virtioscsi0,bus=pci.3,addr=0x1,iothread=iothread-virtioscsi0' -drive 'file=/dev/pve/vm-101-disk-0,if=none,id=drive-scsi0,format=raw,cache=none,aio=io_uring,detect-zeroes=on' -device 'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=101' -netdev 'type=tap,id=net0,ifname=tap101i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=66:13:30:B5:C3:37,netdev=net0,bus=pci.0,addr=0x12,id=net0,rx_queue_size=1024,tx_queue_size=1024,bootindex=102' -machine 'type=q35+pve0' -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'' failed: got timeout

trying to acquire lock...
TASK ERROR: can't lock file '/var/lock/qemu-server/lock-101.conf' - got timeout

VM 101 qmp command 'set_password' failed - unable to connect to VM 101 qmp socket - timeout after 47 retries
TASK ERROR: Failed to run vncproxy.

Like I mentioned, when I did this removing the pci device, install works

Thanks in advance

 

[leesteken]

If there are no errors or messages about why the VM cannot start in journalctl then the timeout is because you don't have enough free memory. Ballooning and KSM don't work with passthrough. Try starting the VM with much less memory and if that does not help search journalctl for any possible clues.

 

[jeffs42885]

If there are no errors or messages about why the VM cannot start in journalctl then the timeout is because you don't have enough free memory. Ballooning and KSM don't work with passthrough. Try starting the VM with much less memory and if that does not help search journalctl for any possible clues.

Tons of these errors, then the timeout:

Jun 03 14:45:44 proxmox kernel: pcieport 0000:00:02.0: AER: device recovery successful
Jun 03 14:45:44 proxmox kernel: pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.0
Jun 03 14:45:44 proxmox kernel: pcieport 0000:00:02.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)
Jun 03 14:45:44 proxmox kernel: pcieport 0000:00:02.0:   device [8086:2f04] error status/mask=00004000/00000000
Jun 03 14:45:44 proxmox kernel: pcieport 0000:00:02.0:    [14] CmpltTO                (First)
Jun 03 14:45:44 proxmox kernel: pcieport 0000:00:02.0: AER: device recovery successful
Jun 03 14:45:44 proxmox kernel: pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:03:00.0
Jun 03 14:45:44 proxmox kernel: pcieport 0000:00:02.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)
Jun 03 14:45:44 proxmox kernel: pcieport 0000:00:02.0:   device [8086:2f04] error status/mask=00004000/00000000
Jun 03 14:45:44 proxmox kernel: pcieport 0000:00:02.0:    [14] CmpltTO                (First)

 

[jeffs42885]

I was able to resolve this. Go me.

I had to set-

/etc/default/grub

to-

GRUB_CMDLINE_LINUX_DEFAULT="pci=nommconf quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:off,efifb:off"

Turn ACPI off in the options, and make sure that my EFI partition was not using secure boot.

 

[Lams364]

Hi there!

Would it be possible to post your vm configuration file ?
I got exactly the same hardware as you and the same goal.

I ran into your exact problem with the log file filing the host space.
I also have the same error when trying to run the vm.

 

[leesteken]

GRUB_CMDLINE_LINUX_DEFAULT="pci=nommconf quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:off,efifb:off"

nofb nomodeset video=vesafb:off,efifb:off does not work for recents Proxmox versions; you need this work-around when doing passthrough of the boot (or only) GPU. That will fix the BAR cannot reserve mem error.