Vm's won't start most of the time

Jsingh

Well-Known Member
Oct 23, 2018
33
0
46
I am starting with a new proxmox installation with the following configuration:

Threadripper 1950X
Gigabyte X399 Auros Extreme
Geforce Gtx 750
Geforce Gtx 9800 Gtx+
2 X 500 Gb Adata SX8200 Pro
3 X 1Tb HDD for daily backup

I have created two VM's each having one of the cards passed through it. I am facing two issues:

1. The 9800 GTX gives code 43 error even after passing the ROM File.
2. The VMs don't start 70% of the time mostly when running together

The error I get is : TASK ERROR: start failed:...... failed: got a timeout

or sometimes I proxmox will show that there is an internal error in the VM
 
I get this and this only happens when I pass through my second GPU:

start failed: command '/usr/bin/kvm -id 101 -name DesignVm2 -chardev 'socket,id=qmp,path=/var/run/qemu-server/101.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/101.pid -daemonize -smbios 'type=1,uuid=0c286fb4-0940-4476-b331-a4173f6798dc' -drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' -drive 'if=pflash,unit=1,format=qcow2,id=drive-efidisk0,file=/mnt/VmStore/images/101/vm-101-disk-1.qcow2' -smp '6,sockets=1,cores=6,maxcpus=6' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga none -nographic -no-hpet -cpu 'host,+pcid,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,kvm=off' -m 8192 -object 'memory-backend-ram,id=ram-node0,size=8192M' -numa 'node,nodeid=0,cpus=0-5,memdev=ram-node0' -object 'iothread,id=iothread-virtioscsi0' -device 'vmgenid,guid=a7cc9349-1a65-4f2e-a88d-4d89082092b9' -readconfig /usr/share/qemu-server/pve-q35.cfg -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=0a:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0' -chardev 'socket,path=/var/run/qemu-server/101.qga,server,nowait,id=qga0' -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:19663b5e5e4b' -device 'virtio-scsi-pci,id=virtioscsi0,bus=pci.3,addr=0x1,iothread=iothread-virtioscsi0' -drive 'file=/mnt/VmStore/images/101/vm-101-disk-0.qcow2,if=none,id=drive-scsi0,cache=writeback,discard=on,format=qcow2,aio=threads,detect-zeroes=unmap' -device 'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,rotation_rate=1,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap101i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=6E:39:1C:C8:84:A1,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -rtc 'driftfix=slew,base=localtime' -machine 'type=q35' -global 'kvm-pit.lost_tick_policy=discard' -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'' failed: got timeout​
 
Run this comment directly to console (ssh).
This is the same command I just removed daemonize option
/usr/bin/kvm \
-id 101 \
-name DesignVm2 \
-chardev 'socket,id=qmp,path=/var/run/qemu-server/101.qmp,server,nowait' \
-mon 'chardev=qmp,mode=control' \
-chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \
-mon 'chardev=qmp-event,mode=control' \
-pidfile /var/run/qemu-server/101.pid \
-smbios 'type=1,uuid=0c286fb4-0940-4476-b331-a4173f6798dc' \
-drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' \
-drive 'if=pflash,unit=1,format=qcow2,id=drive-efidisk0,file=/mnt/VmStore/images/101/vm-101-disk-1.qcow2' \
-smp '6,sockets=1,cores=6,maxcpus=6' \
-nodefaults \
-boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
-vga none \
-nographic \
-no-hpet \
-cpu 'host,+pcid,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,kvm=off' \
-m 8192 \
-object 'memory-backend-ram,id=ram-node0,size=8192M' \
-numa 'node,nodeid=0,cpus=0-5,memdev=ram-node0' \
-object 'iothread,id=iothread-virtioscsi0' \
-device 'vmgenid,guid=a7cc9349-1a65-4f2e-a88d-4d89082092b9' \
-readconfig /usr/share/qemu-server/pve-q35.cfg \
-device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \
-device 'vfio-pci,host=0a:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0' \
-chardev 'socket,path=/var/run/qemu-server/101.qga,server,nowait,id=qga0' \
-device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' \
-device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' \
-device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \
-iscsi 'initiator-name=iqn.1993-08.org.debian:01:19663b5e5e4b' \
-device 'virtio-scsi-pci,id=virtioscsi0,bus=pci.3,addr=0x1,iothread=iothread-virtioscsi0' \
-drive 'file=/mnt/VmStore/images/101/vm-101-disk-0.qcow2,if=none,id=drive-scsi0,cache=writeback,discard=on,format=qcow2,aio=threads,detect-zeroes=unmap' \
-device 'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,rotation_rate=1,bootindex=100' \
-netdev 'type=tap,id=net0,ifname=tap101i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' \
-device 'virtio-net-pci,mac=6E:39:1C:C8:84:A1,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' \
-rtc 'driftfix=slew,base=localtime' \
-machine 'type=q35' \
-global 'kvm-pit.lost_tick_policy=discard' \
-cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off
 
Don`t know why Proxmox gets timeout but you can add daemonize and KVM will run in background.

/usr/bin/kvm \
-id 101 \
-name DesignVm2 \
-chardev 'socket,id=qmp,path=/var/run/qemu-server/101.qmp,server,nowait' \
-mon 'chardev=qmp,mode=control' \
-chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \
-mon 'chardev=qmp-event,mode=control' \
-pidfile /var/run/qemu-server/101.pid \
-daemonize \
-smbios 'type=1,uuid=0c286fb4-0940-4476-b331-a4173f6798dc' \
-drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' \
-drive 'if=pflash,unit=1,format=qcow2,id=drive-efidisk0,file=/mnt/VmStore/images/101/vm-101-disk-1.qcow2' \
-smp '6,sockets=1,cores=6,maxcpus=6' \
-nodefaults \
-boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
-vga none \
-nographic \
-no-hpet \
-cpu 'host,+pcid,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,kvm=off' \
-m 8192 \
-object 'memory-backend-ram,id=ram-node0,size=8192M' \
-numa 'node,nodeid=0,cpus=0-5,memdev=ram-node0' \
-object 'iothread,id=iothread-virtioscsi0' \
-device 'vmgenid,guid=a7cc9349-1a65-4f2e-a88d-4d89082092b9' \
-readconfig /usr/share/qemu-server/pve-q35.cfg \
-device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \
-device 'vfio-pci,host=0a:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0' \
-chardev 'socket,path=/var/run/qemu-server/101.qga,server,nowait,id=qga0' \
-device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' \
-device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' \
-device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \
-iscsi 'initiator-name=iqn.1993-08.org.debian:01:19663b5e5e4b' \
-device 'virtio-scsi-pci,id=virtioscsi0,bus=pci.3,addr=0x1,iothread=iothread-virtioscsi0' \
-drive 'file=/mnt/VmStore/images/101/vm-101-disk-0.qcow2,if=none,id=drive-scsi0,cache=writeback,discard=on,format=qcow2,aio=threads,detect-zeroes=unmap' \
-device 'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,rotation_rate=1,bootindex=100' \
-netdev 'type=tap,id=net0,ifname=tap101i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' \
-device 'virtio-net-pci,mac=6E:39:1C:C8:84:A1,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' \
-rtc 'driftfix=slew,base=localtime' \
-machine 'type=q35' \
-global 'kvm-pit.lost_tick_policy=discard' \
-cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off
 
Don`t know why Proxmox gets timeout but you can add daemonize and KVM will run in background.
we just have timeout logic for a lot of our commands (start, stop etc.) if qemu doesn't respond or something like that (also present in lxc pct commands for example).

i might look into a fix for this.. can you suggest anything maybe?
 
we already have that in most of them with --timeout option on cli
 
So I replaced the 9800 GTX with a geforce 1050Ti so now both Cards are UEFI compatible. I passed through both cards to different VMs and Windows installed the Basic Display adapter sucessfully.

Then I installed the driver and the moment I installed the VGA driver, the VM with Geforce 750(set as boot Graphics device in the bios installed in PCIe slot 1) froze. When I tried to stop the VM from the VM Manager the host crashed.

I restarted the host now the second VM won't start when the first is started giving a timeout error.

Don`t know why Proxmox gets timeout but you can add daemonize and KVM will run in background.

How can I do that? The Virtual Machines will be used by specific users who are not technical so I need to be able to start the VM from GUI.
 
Last edited:
[Update] So I had enabled memory balloon in both of Vm's. So My config was:

VM1: min: 8Gb max: 8gb
VM2: min: 4Gb max: 8Gb

I changed the min on Vm2 to 8 Gb and the Vm's are behaving correctly now.
My theory is:
1. The non uefi 9800 GTX was causing the timeout issues initially.
2. The memory balloon was causing instability and timeout errors.

Is this a known issue? Will it create issues when I pass through the SATA controller and a nvme controller to another VM I am planning to deploy?

I am going to watch the stability of the system over the next few days and see how it behaves.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!