I can't get a proxmox machine to start.

[Kubernat]

Hi.

For a few days now, I've had a machine that I can't get to boot. This machine has two storage units: one for the system and the other for most of the data, which is a zpool.

I removed the zpool because I wasn't sure if it could be a fault with this storage unit, but the machine still won't boot.
The error we're seeing is the following:

TASK ERROR: timeout waiting on systemd

The only time my machine starts is when I restart proxmox but it doesn't finish starting, and proxmox itself closes it and I can't start it anymore.

Any help? Any commands to send more information?

The most important thing is to bring up the machine and reattach the zpool storage I removed earlier.

Thank you very much.

 

[l.leahu-vladucu]

Hello Kubernat! Just to be sure I understand correctly, you are talking about not being able to start a VM, and not Proxmox VE itself, right?

In this case, please provide us with the following:

1. The output of pveversion -v
2. The full task log when trying to start the VM (in case it says more than "TASK ERROR: timeout waiting on systemd")
3. A journal of the system around the time the issues started to happen, with journalctl --since <TIME> --until <TIME> > journal.txt

 

[Kubernat]

Hello l.leahu-vladucu

Thaks for the help, I tried to send you everything you asked for.

1: execute the first command
pveversion -v
proxmox-ve: 8.4.0 (running kernel: 6.8.12-10-pve)
pve-manager: 8.4.1 (running version: 8.4.1/2a5fa54a8503f96d)
proxmox-kernel-helper: 8.1.1
pve-kernel-5.15: 7.4-13
proxmox-kernel-6.8.12-10-pve-signed: 6.8.12-10
proxmox-kernel-6.8: 6.8.12-10
proxmox-kernel-6.8.4-3-pve-signed: 6.8.4-3
pve-kernel-5.15.152-1-pve: 5.15.152-1
pve-kernel-5.4.203-1-pve: 5.4.203-1
pve-kernel-4.15: 5.4-9
pve-kernel-4.15.18-21-pve: 4.15.18-48
pve-kernel-4.15.18-9-pve: 4.15.18-30
ceph: 16.2.15+ds-0+deb12u1
ceph-fuse: 16.2.15+ds-0+deb12u1
corosync: 3.1.9-pve1
criu: 3.17.1-2+deb12u1
glusterfs-client: 10.3-5
ifupdown: residual config
ifupdown2: 3.2.0-1+pmx11
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libknet1: 1.30-pve2
libproxmox-acme-perl: 1.6.0
libproxmox-backup-qemu0: 1.5.1
libproxmox-rs-perl: 0.3.5
libpve-access-control: 8.2.2
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.1.0
libpve-cluster-perl: 8.1.0
libpve-common-perl: 8.3.1
libpve-guest-common-perl: 5.2.2
libpve-http-server-perl: 5.2.2
libpve-network-perl: 0.11.2
libpve-rs-perl: 0.9.4
libpve-storage-perl: 8.3.6
libqb0: 1.0.5-1
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.6.0-2
proxmox-backup-client: 3.4.1-1
proxmox-backup-file-restore: 3.4.1-1
proxmox-firewall: 0.7.1
proxmox-kernel-helper: 8.1.1
proxmox-mail-forward: 0.3.2
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.7
proxmox-widget-toolkit: 4.3.10
pve-cluster: 8.1.0
pve-container: 5.2.6
pve-docs: 8.4.0
pve-edk2-firmware: 4.2025.02-3
pve-esxi-import-tools: 0.7.4
pve-firewall: 5.1.1
pve-firmware: 3.15-3
pve-ha-manager: 4.0.7
pve-i18n: 3.4.2
pve-qemu-kvm: 9.2.0-5
pve-xtermjs: 5.5.0-2
qemu-server: 8.3.12
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.7-pve2


2: I run journalctl -xe | grep qm and I leave you the result of the interval between 12:40 and 12:41, which is when I tried to start the machine.


May 13 12:40:07 Node01 pvedaemon[128953]: start VM 300: UPID:Node01:0001F7B9:0014D02D:68232187:qmstart:300:user:
May 13 12:40:07 Node01 pvedaemon[121439]: <user> starting task UPID:Node01:0001F7B9:0014D02D:68232187:qmstart:300:user:
May 13 12:40:28 Node01 pvedaemon[121439]: <user> end task UPID:Node01:0001F7B9:0014D02D:68232187:qmstart:300:user: timeout waiting on systemd



3: I try tu start te VM


journalctl --since=12:39:00 --until=12:41:00 > journal.txt

May 13 12:39:16 Node01 zed[1456]: Missed 144 events
May 13 12:39:16 Node01 zed[128528]: eid=195 class=deadman pool='zpool_storage' vdev=scsi-36141877048c0a500286c3fc10e4aa1e3-part1 size=933888 offset=238720450560 priority=4 err=0 flags=0x400804b0 delay=7320059ms
May 13 12:39:28 Node01 zed[1456]: Missed 14 events
May 13 12:39:28 Node01 zed[128616]: eid=196 class=deadman pool='zpool_storage' vdev=scsi-36141877048c0a500286c3fc10e4aa1e3-part1 size=4096 offset=238721380352 priority=4 err=0 flags=0x3800b0 bookmark=2051:1:0:1216398745
May 13 12:40:07 Node01 pvedaemon[128953]: start VM 300: UPID:Node01:0001F7B9:0014D02D:68232187:qmstart:300:user:
May 13 12:40:07 Node01 pvedaemon[121439]: <user> starting task UPID:Node01:0001F7B9:0014D02D:68232187:qmstart:300:user:
May 13 12:40:17 Node01 zed[1456]: Missed 159 events
May 13 12:40:17 Node01 zed[129050]: eid=197 class=deadman pool='zpool_storage' vdev=scsi-36141877048c0a500286c3fc10e4aa1e3-part1 size=933888 offset=238720450560 priority=4 err=0 flags=0x400804b0 delay=7320059ms
May 13 12:40:28 Node01 pvedaemon[128953]: timeout waiting on systemd
May 13 12:40:28 Node01 pvedaemon[121439]: <user> end task UPID:Node01:0001F7B9:0014D02D:68232187:qmstart:300:user: timeout waiting on systemd
May 13 12:40:29 Node01 zed[1456]: Missed 29 events
May 13 12:40:29 Node01 zed[129136]: eid=198 class=deadman pool='zpool_storage' vdev=scsi-36141877048c0a500286c3fc10e4aa1e3-part1 size=4096 offset=238721380352 priority=4 err=0 flags=0x3800b0 bookmark=2051:1:0:1216398745

Thank you so much