VM doesn't start anymore

S

svendaems

New Member
Dec 24, 2025
6
0
1
I had a simple Debian VM installed and that was running fine. After a while, the VM crashed and I can't boot it anymore. I get following error

Code: 
root@proxmox:/# qm start 201
start failed: command '/usr/bin/kvm -id 201 -name 'dockerhost01,debug-threads=on' -no-shutdown -chardev 'socket,id=qmp,path=/var/run/qemu-
server/201.qmp,server=on,wait=off' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect-ms=5000' -mon 'chardev=qmp-
event,mode=control' -pidfile /var/run/qemu-server/201.pid -daemonize -smbios 'type=1,uuid=bbec40bc-3406-46bb-9f75-7826498c56c3' -smp '8,sockets=1,cores=8,maxcpus=8' -
nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vnc 'unix:/var/run/qemu-server/201.vnc,password=on' -cpu 
qemu64,+aes,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+pni,+popcnt,+sse4.1,+sse4.2,+ssse3 -m 4096 -object '{"id":"throttle-drive-ide2","limits":{},"qom-type":"throttle-group"}' -object 
'iothread,id=iothread-virtioscsi0' -object '{"id":"throttle-drive-scsi0","limits":{},"qom-type":"throttle-group"}' -global 'PIIX4_PM.disable_s3=1' -global 'PIIX4_PM.disable_s4=1' -device 
'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pci.0,addr=0x5' -
device 'vmgenid,guid=392595c7-4d39-4907-a986-3910e68eadc6' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 
'VGA,id=vga,bus=pci.0,addr=0x2' -chardev 'socket,path=/var/run/qemu-server/201.qga,server=on,wait=off,id=qga0' -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' -device 
'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' -iscsi 'initiator-name=iqn.1993-
08.org.debian:01:e01d7a1b517' -blockdev '{"driver":"throttle","file":{"cache":{"direct":false,"no-flush":false},"driver":"raw","file":{"aio":"io_uring","cache":{"direct":false,"no-
flush":false},"driver":"file","filename":"/var/lib/vz/template/iso/debian-13.2.0-amd64-netinst.iso","node-name":"e5db143f1419ad3166b881830cde3b2","read-only":true},"node-
name":"f5db143f1419ad3166b881830cde3b2","read-only":true},"node-name":"drive-ide2","read-only":true,"throttle-group":"throttle-drive-ide2"}' -device 'ide-
cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=101' -device 'virtio-scsi-pci,id=virtioscsi0,bus=pci.3,addr=0x1,iothread=iothread-virtioscsi0' -blockdev '{"detect-
zeroes":"on","discard":"ignore","driver":"throttle","file":{"cache":{"direct":true,"no-flush":false},"detect-zeroes":"on","discard":"ignore","driver":"qcow2","file":
{"aio":"io_uring","cache":{"direct":true,"no-flush":false},"detect-zeroes":"on","discard":"ignore","driver":"file","filename":"/mnt/pve/Synology/images/201/vm-201-disk-
0.qcow2","node-name":"eb09f824ea4b160447db38cd0a962fe","read-only":false},"node-name":"fb09f824ea4b160447db38cd0a962fe","read-only":false},"node-name":"drive-
scsi0","read-only":false,"throttle-group":"throttle-drive-scsi0"}' -device 'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,device_id=drive-
scsi0,bootindex=100,write-cache=on' -netdev 'type=tap,id=net0,ifname=tap201i0,script=/usr/libexec/qemu-server/pve-bridge,downscript=/usr/libexec/qemu-server/pve-
bridgedown,vhost=on' -device 'virtio-net-
pci,mac=BC:24:11:BC:52:C5,netdev=net0,bus=pci.0,addr=0x12,id=net0,rx_queue_size=1024,tx_queue_size=256,bootindex=102,host_mtu=1500' -machine 'hpet=off,type=pc+pve0'' 
failed: got timeout

This is how the config looks like
Code: 
root@proxmox:/# qm config 201
agent: 1
boot: order=scsi0;ide2;net0
cores: 8
cpu: x86-64-v2-AES
ide2: local:iso/debian-13.2.0-amd64-netinst.iso,media=cdrom,size=784M
memory: 4096
meta: creation-qemu=10.1.2,ctime=1765288023
name: dockerhost01
net0: virtio=BC:24:11:BC:52:C5,bridge=vmbr0
numa: 0
onboot: 1
ostype: l26
scsi0: Synology:201/vm-201-disk-0.qcow2,iothread=1,size=32G
scsihw: virtio-scsi-single
smbios1: uuid=bbec40bc-3406-46bb-9f75-7826498c56c3
sockets: 1
vmgenid: 392595c7-4d39-4907-a986-3910e68eadc6

I've searched but I can't find any logs (no /var/log/syslog file)

This is more information from my host
Code: 
root@proxmox:/# pveversion -v
proxmox-ve: 9.1.0 (running kernel: 6.17.2-2-pve)
pve-manager: 9.1.2 (running version: 9.1.2/9d436f37a0ac4172)
proxmox-kernel-helper: 9.0.4
proxmox-kernel-6.17.2-2-pve-signed: 6.17.2-2
proxmox-kernel-6.17: 6.17.2-2
proxmox-kernel-6.17.2-1-pve-signed: 6.17.2-1
amd64-microcode: 3.20250311.1
ceph-fuse: 19.2.3-pve2
corosync: 3.1.9-pve2
criu: 4.1.1-1
frr-pythontools: 10.4.1-1+pve1
ifupdown2: 3.3.0-1+pmx11
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libproxmox-acme-perl: 1.7.0
libproxmox-backup-qemu0: 2.0.1
libproxmox-rs-perl: 0.4.1
libpve-access-control: 9.0.4
libpve-apiclient-perl: 3.4.2
libpve-cluster-api-perl: 9.0.7
libpve-cluster-perl: 9.0.7
libpve-common-perl: 9.1.0
libpve-guest-common-perl: 6.0.2
libpve-http-server-perl: 6.0.5
libpve-network-perl: 1.2.3
libpve-rs-perl: 0.11.3
libpve-storage-perl: 9.1.0
libspice-server1: 0.15.2-1+b1
lvm2: 2.03.31-2+pmx1
lxc-pve: 6.0.5-3
lxcfs: 6.0.4-pve1
novnc-pve: 1.6.0-3
proxmox-backup-client: 4.1.0-1
proxmox-backup-file-restore: 4.1.0-1
proxmox-backup-restore-image: 1.0.0
proxmox-firewall: 1.2.1
proxmox-kernel-helper: 9.0.4
proxmox-mail-forward: 1.0.2
proxmox-mini-journalreader: 1.6
proxmox-offline-mirror-helper: 0.7.3
proxmox-widget-toolkit: 5.1.2
pve-cluster: 9.0.7
pve-container: 6.0.18
pve-docs: 9.1.1
pve-edk2-firmware: 4.2025.05-2
pve-esxi-import-tools: 1.0.1
pve-firewall: 6.0.4
pve-firmware: 3.17-2
pve-ha-manager: 5.0.8
pve-i18n: 3.6.5
pve-qemu-kvm: 10.1.2-4
pve-xtermjs: 5.5.0-3
qemu-server: 9.1.1
smartmontools: 7.4-pve1
spiceterm: 3.4.1
swtpm: 0.8.0+pve3
vncterm: 1.9.1
zfsutils-linux: 2.3.4-pve1

Anybody an idea how I can see more log information or an idea why the VM won't start?
 
If there is no actual error message and only a time out then it's almost always not enough memory free for the VM. Check the System Log in the Proxmox web GUI or journalctl from around the time that you tried to start the VM.
 
  • Like
Reactions: svendaems
The log files in journalctl didn't learned me anything

Code: 
Dec 24 17:35:10 proxmox pvedaemon[1325084]: start failed: command '/usr/bin/kvm -id 201 -name 'dockerhost01,debug-threads=on' -no-shutdown -chardev 'socket,id=qmp,path=/var/run/qemu-server/201.qmp,server=on,wait=off' -mon 'chardev=qmp,mode=control' -chard>
Dec 24 17:35:10 proxmox pvedaemon[3109207]: <root@pam> end task UPID:proxmox:0014381C:09B2BAD4:694C161B:qmstart:201:root@pam: start failed: command '/usr/bin/kvm -id 201 -name 'dockerhost01,debug-threads=on' -no-shutdown -chardev 'socket,id=qmp,path=/var/>

I've ended up rebooting my Proxmox host and this solved the issue. Lets hope it stays stable now
 
I've only 12GB of my 32 GB assigned to VM's, the rest is not allocated.

I've checked , there zfs_arc_max was set to 3020947456, so I guess this is around 2GB
 
The is no other information in the little snippet of the log that you showed (and it was also cut off on the right. What other (whole) messages were there during the minute or so before (I don't know how long the time-out is)?
Do you have some swap space? Otherwise memory fragmentation might (over time) cause too little (continuous) memory to be available for VMs.
svendaems said:
I've only 12GB of my 32 GB assigned to VM's, the rest is not allocated.
Click to expand...
Is the maximum memory set for all the VMs together 12GB or does it look like they are using only 12GB?
 
There was only 12GB allocated, 16GB was unallocated. I even extended memory from 2 to 4GB of the broken VM but this also didn't helped.
 
Another thing that may be worth looking at is the bootdisk of the VM. I noticed that it appears to reside in Synology. Is this an NFS/SMB/CIFS share? You maybe having issues with NW in this regard. If you try migrating it to a local storage - do things improve?
 
It's an NFS share. Another VM that was running fine was also located on that share, but it's a good tip to try it next time if it would happen again.
 
svendaems said:
Another VM that was running fine was also located on that share
Click to expand...
I notice the (problematic) VM appears to run Docker. Depending on its complexity of containers/NW setup - it maybe somewhat overwhelming your NW - which would aggravate the NFS share. Possibly the working VM was a simpler affair.
 
  • Like
Reactions: leesteken
Both working VM's are docker hosts, but both have for now only 1 container running & it's on both the same container :-D
 
You must log in or register to reply here.
Top Bottom