VM constantly times out when starting up

PVEboi69

New Member
Apr 29, 2020
6
0
1
24
Hello! I am attempting to start up one of my VMs (107) and it is constantly timing out when I try to start it. It has a GTX 780 and USB card passed through to it, and it is stored on a local ZFS array on 10k spinner drives. It looks like it is having issues with the Vnic as it is starting up, because the syslog is referencing a port that is on that VM as it is starting up. I have attached a copy of the config of the vm. Below is the syslog when I attempt to start the VM. Here it is below:

Code:
Jun 20 17:08:24 pve2 pvedaemon[5927]: <root@pam> update VM 107: -net0 virtio,bridge=vmbr1,firewall=1
Jun 20 17:08:26 pve2 pvedaemon[5926]: <root@pam> starting task UPID:pve2:00001A97:0131F533:60CFBC5A:qmstart:107:root@pam:
Jun 20 17:08:26 pve2 pvedaemon[6807]: start VM 107: UPID:pve2:00001A97:0131F533:60CFBC5A:qmstart:107:root@pam:
Jun 20 17:08:28 pve2 systemd[1]: Started 107.scope.
Jun 20 17:08:28 pve2 systemd-udevd[6824]: Using default interface naming scheme 'v240'.
Jun 20 17:08:28 pve2 systemd-udevd[6824]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jun 20 17:08:28 pve2 systemd-udevd[6824]: Could not generate persistent MAC address for tap107i0: No such file or directory
Jun 20 17:08:29 pve2 kernel: device tap107i0 entered promiscuous mode
Jun 20 17:08:29 pve2 systemd-udevd[6824]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jun 20 17:08:29 pve2 systemd-udevd[6824]: Could not generate persistent MAC address for fwbr107i0: No such file or directory
Jun 20 17:08:29 pve2 systemd-udevd[6828]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jun 20 17:08:29 pve2 systemd-udevd[6828]: Using default interface naming scheme 'v240'.
Jun 20 17:08:29 pve2 systemd-udevd[6828]: Could not generate persistent MAC address for fwpr107p0: No such file or directory
Jun 20 17:08:29 pve2 systemd-udevd[6825]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jun 20 17:08:29 pve2 systemd-udevd[6825]: Using default interface naming scheme 'v240'.
Jun 20 17:08:29 pve2 systemd-udevd[6825]: Could not generate persistent MAC address for fwln107i0: No such file or directory
Jun 20 17:08:29 pve2 kernel: fwbr107i0: port 1(fwln107i0) entered blocking state
Jun 20 17:08:29 pve2 kernel: fwbr107i0: port 1(fwln107i0) entered disabled state
Jun 20 17:08:29 pve2 kernel: device fwln107i0 entered promiscuous mode
Jun 20 17:08:29 pve2 kernel: fwbr107i0: port 1(fwln107i0) entered blocking state
Jun 20 17:08:29 pve2 kernel: fwbr107i0: port 1(fwln107i0) entered forwarding state
Jun 20 17:08:29 pve2 kernel: vmbr1: port 2(fwpr107p0) entered blocking state
Jun 20 17:08:29 pve2 kernel: vmbr1: port 2(fwpr107p0) entered disabled state
Jun 20 17:08:29 pve2 kernel: device fwpr107p0 entered promiscuous mode
Jun 20 17:08:29 pve2 kernel: vmbr1: port 2(fwpr107p0) entered blocking state
Jun 20 17:08:29 pve2 kernel: vmbr1: port 2(fwpr107p0) entered forwarding state
Jun 20 17:08:29 pve2 kernel: fwbr107i0: port 2(tap107i0) entered blocking state
Jun 20 17:08:29 pve2 kernel: fwbr107i0: port 2(tap107i0) entered disabled state
Jun 20 17:08:29 pve2 kernel: fwbr107i0: port 2(tap107i0) entered blocking state
Jun 20 17:08:29 pve2 kernel: fwbr107i0: port 2(tap107i0) entered forwarding state
Jun 20 17:08:35 pve2 pvedaemon[5927]: VM 107 qmp command failed - VM 107 qmp command 'query-proxmox-support' failed - got timeout
Jun 20 17:08:42 pve2 pvestatd[5902]: VM 107 qmp command failed - VM 107 qmp command 'query-proxmox-support' failed - unable to connect to VM 107 qmp socket - timeout after 31 retries
Jun 20 17:08:43 pve2 pvestatd[5902]: status update time (6.929 seconds)
Jun 20 17:08:52 pve2 pvestatd[5902]: VM 107 qmp command failed - VM 107 qmp command 'query-proxmox-support' failed - unable to connect to VM 107 qmp socket - timeout after 31 retries
Jun 20 17:08:53 pve2 pvestatd[5902]: status update time (7.136 seconds)
Jun 20 17:08:58 pve2 pvedaemon[6807]: start failed: command '/usr/bin/kvm -id 107 -name BenVDI -no-shutdown -chardev 'socket,id=qmp,path=/var/run/qemu-server/107.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/107.pid -daemonize -smbios 'type=1,uuid=36819c94-3dc3-4142-9f21-b5cd72adb4d5' -drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' -drive 'if=pflash,unit=1,format=raw,id=drive-efidisk0,size=131072,file=/dev/zvol/PVE2-VDI-STORE/vm-107-disk-1' -smp '20,sockets=1,cores=20,maxcpus=20' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga none -nographic -no-hpet -cpu 'host,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vpindex,+kvm_pv_eoi,+kvm_pv_unhalt' -m 16384 -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg -device 'vmgenid,guid=1c382b30-d5be-4759-b01a-4e82463c339d' -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=0000:05:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on' -device 'vfio-pci,host=0000:05:00.1,id=hostpci0.1,bus=ich9-pcie-port-1,addr=0x0.1' -device 'vfio-pci,host=0000:41:00.0,id=hostpci1,bus=ich9-pcie-port-2,addr=0x0,rombar=0' -chardev 'socket,path=/var/run/qemu-server/107.qga,server,nowait,id=qga0' -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:1e489ed0f878' -drive 'file=/mnt/pve/PVE-CENT/template/iso/virtio.iso,if=none,id=drive-ide0,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.0,unit=0,drive=drive-ide0,id=ide0' -device 'virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5' -drive 'file=/dev/zvol/PVE2-VDI-STORE/vm-107-disk-0,if=none,id=drive-scsi0,cache=writeback,format=raw,aio=threads,detect-zeroes=on' -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap107i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=FA:E4:FB:A4:CB:4E,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=101' -rtc 'driftfix=slew,base=localtime' -machine 'type=pc-q35-5.1+pve0' -global 'kvm-pit.lost_tick_policy=discard'' failed: got timeout
Jun 20 17:08:58 pve2 pvedaemon[5926]: <root@pam> end task UPID:pve2:00001A97:0131F533:60CFBC5A:qmstart:107:root@pam: start failed: command '/usr/bin/kvm -id 107 -name BenVDI -no-shutdown -chardev 'socket,id=qmp,path=/var/run/qemu-server/107.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/107.pid -daemonize -smbios 'type=1,uuid=36819c94-3dc3-4142-9f21-b5cd72adb4d5' -drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' -drive 'if=pflash,unit=1,format=raw,id=drive-efidisk0,size=131072,file=/dev/zvol/PVE2-VDI-STORE/vm-107-disk-1' -smp '20,sockets=1,cores=20,maxcpus=20' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga none -nographic -no-hpet -cpu 'host,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vpindex,+kvm_pv_eoi,+kvm_pv_unhalt' -m 16384 -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg -device 'vmgenid,guid=1c382b30-d5be-4759-b01a-4e82463c339d' -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=0000:05:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on' -device 'vfio-pci,host=0000:05:00.1,id=hostpci0.1,bus=ich9-pcie-port-1,addr=0x0.1' -device 'vfio-pci,host=0000:41:00.0,id=hostpci1,bus=ich9-pcie-port-2,addr=0x0,rombar=0' -chardev 'socket,path=/var/run/qemu-server/107.qga,server,nowait,id=qga0' -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:1e489ed0f878' -drive 'file=/mnt/pve/PVE-CENT/template/iso/virtio.iso,if=none,id=drive-ide0,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.0,unit=0,drive=drive-ide0,id=ide0' -device 'virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5' -drive 'file=/dev/zvol/PVE2-VDI-STORE/vm-107-disk-0,if=none,id=drive-scsi0,cache=writeback,format=raw,aio=threads,detect-zeroes=on' -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap107i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=FA:E4:FB:A4:CB:4E,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=101' -rtc 'driftfix=slew,base=localtime' -machine 'type=pc-q35-5.1+pve0' -global 'kvm-pit.lost_tick_policy=discard'' failed: got timeout
Jun 20 17:09:00 pve2 systemd[1]: Starting Proxmox VE replication runner...
Jun 20 17:09:00 pve2 systemd[1]: pvesr.service: Succeeded.
Jun 20 17:09:00 pve2 systemd[1]: Started Proxmox VE replication runner.
Jun 20 17:09:00 pve2 kernel: vfio-pci 0000:05:00.0: vfio_ecap_init: hiding ecap 0x19@0x900
Jun 20 17:09:02 pve2 pvestatd[5902]: VM 107 qmp command failed - VM 107 qmp command 'query-proxmox-support' failed - unable to connect to VM 107 qmp socket - timeout after 31 retries
Jun 20 17:09:03 pve2 pvestatd[5902]: status update time (6.350 seconds)
Jun 20 17:09:05 pve2 kernel: fwbr107i0: port 2(tap107i0) entered disabled state
Jun 20 17:09:05 pve2 kernel: fwbr107i0: port 1(fwln107i0) entered disabled state
Jun 20 17:09:05 pve2 kernel: vmbr1: port 2(fwpr107p0) entered disabled state
Jun 20 17:09:05 pve2 kernel: device fwln107i0 left promiscuous mode
Jun 20 17:09:05 pve2 kernel: fwbr107i0: port 1(fwln107i0) entered disabled state
Jun 20 17:09:06 pve2 kernel: device fwpr107p0 left promiscuous mode
Jun 20 17:09:06 pve2 kernel: vmbr1: port 2(fwpr107p0) entered disabled state
Jun 20 17:09:06 pve2 pvedaemon[5927]: VM 107 qmp command failed - VM 107 qmp command 'query-proxmox-support' failed - unable to connect to VM 107 qmp socket - timeout after 31 retries
Jun 20 17:09:06 pve2 pvestatd[5902]: VM 107 qmp command failed - VM 107 not running
Jun 20 17:09:10 pve2 systemd[1]: 107.scope: Succeeded.
Jun 20 17:09:32 pve2 pvedaemon[5925]: <root@pam> update VM 107: -net0 virtio=FA:E4:FB:A4:CB:4E,bridge=vmbr1
Jun 20 17:09:34 pve2 pvedaemon[5926]: <root@pam> starting task UPID:pve2:00002421:01320F9B:60CFBC9E:qmstart:107:root@pam:
Jun 20 17:09:34 pve2 pvedaemon[9249]: start VM 107: UPID:pve2:00002421:01320F9B:60CFBC9E:qmstart:107:root@pam:
Jun 20 17:09:35 pve2 systemd[1]: Started 107.scope.
Jun 20 17:09:35 pve2 systemd-udevd[9264]: Using default interface naming scheme 'v240'.
Jun 20 17:09:35 pve2 systemd-udevd[9264]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jun 20 17:09:35 pve2 systemd-udevd[9264]: Could not generate persistent MAC address for tap107i0: No such file or directory
Jun 20 17:09:36 pve2 kernel: device tap107i0 entered promiscuous mode
Jun 20 17:09:36 pve2 kernel: vmbr1: port 2(tap107i0) entered blocking state
Jun 20 17:09:36 pve2 kernel: vmbr1: port 2(tap107i0) entered disabled state
Jun 20 17:09:36 pve2 kernel: vmbr1: port 2(tap107i0) entered blocking state
Jun 20 17:09:36 pve2 kernel: vmbr1: port 2(tap107i0) entered forwarding state
Jun 20 17:09:42 pve2 pvedaemon[5925]: VM 107 qmp command failed - VM 107 qmp command 'query-proxmox-support' failed - unable to connect to VM 107 qmp socket - timeout after 31 retries
Jun 20 17:09:42 pve2 pvestatd[5902]: VM 107 qmp command failed - VM 107 qmp command 'query-proxmox-support' failed - unable to connect to VM 107 qmp socket - timeout after 31 retries
Jun 20 17:09:43 pve2 pvestatd[5902]: status update time (6.850 seconds)
Jun 20 17:09:45 pve2 pmxcfs[5769]: [status] notice: received log
Jun 20 17:09:45 pve2 sshd[9513]: Accepted publickey for root from 192.168.5.10 port 46422 ssh2: RSA SHA256:Lm3wEk2MT8DVIDjbptOBF/0oV3KG37TZqTn91wV8V90
Jun 20 17:09:45 pve2 sshd[9513]: pam_unix(sshd:session): session opened for user root by (uid=0)
Jun 20 17:09:45 pve2 systemd[1]: Created slice User Slice of UID 0.
Jun 20 17:09:45 pve2 systemd[1]: Starting User Runtime Directory /run/user/0...
Jun 20 17:09:45 pve2 systemd-logind[5355]: New session 154 of user root.
Jun 20 17:09:45 pve2 systemd[1]: Started User Runtime Directory /run/user/0.
Jun 20 17:09:45 pve2 systemd[1]: Starting User Manager for UID 0...
Jun 20 17:09:45 pve2 systemd[9522]: pam_unix(systemd-user:session): session opened for user root by (uid=0)
Jun 20 17:09:45 pve2 systemd[9522]: Reached target Timers.
Jun 20 17:09:45 pve2 systemd[9522]: Listening on GnuPG cryptographic agent and passphrase cache.
Jun 20 17:09:45 pve2 systemd[9522]: Listening on GnuPG cryptographic agent and passphrase cache (access for web browsers).
Jun 20 17:09:45 pve2 systemd[9522]: Listening on GnuPG cryptographic agent and passphrase cache (restricted).
Jun 20 17:09:45 pve2 systemd[9522]: Reached target Paths.
Jun 20 17:09:45 pve2 systemd[9522]: Listening on GnuPG network certificate management daemon.
Jun 20 17:09:45 pve2 systemd[9522]: Listening on GnuPG cryptographic agent (ssh-agent emulation).
Jun 20 17:09:45 pve2 systemd[9522]: Reached target Sockets.
Jun 20 17:09:45 pve2 systemd[9522]: Reached target Basic System.
Jun 20 17:09:45 pve2 systemd[9522]: Reached target Default.
Jun 20 17:09:45 pve2 systemd[9522]: Startup finished in 98ms.
Jun 20 17:09:45 pve2 systemd[1]: Started User Manager for UID 0.
Jun 20 17:09:45 pve2 systemd[1]: Started Session 154 of user root.
Jun 20 17:09:52 pve2 pvestatd[5902]: VM 107 qmp command failed - VM 107 qmp command 'query-proxmox-support' failed - unable to connect to VM 107 qmp socket - timeout after 31 retries
Jun 20 17:09:53 pve2 pvestatd[5902]: status update time (6.834 seconds)

Thank you for any and all help in advance! If you need any more, feel free to tell me what you need!
 

Attachments

  • image_2021-06-20_171825.png
    image_2021-06-20_171825.png
    31.2 KB · Views: 12
When using PCI passthrough the memory has to be allocated on start. If freeing the memory takes too long, the start of a VM might take too long and it runs into a timeout.

You could try using hugepages.
 
When using PCI passthrough the memory has to be allocated on start. If freeing the memory takes too long, the start of a VM might take too long and it runs into a timeout.

You could try using hugepages.
Pardon my ignorance, but how does one go about enabling said hugepages? I have been looking for a while now and still have not found an answer.
 
You can set the option via `qm set <vmid> --hugepages <value>` where `value` is either `1024`, `2` or `any`.
Setting it to `2` could already help in your situation.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!