Hello I am running Proxmox VE 7.1-10 on Intel NUC8i7HVK. Having issue while trying to reboot Proxmox node, complete proxmox shuts down and never boots up.
Inside Proxmox node I have 1 VM node for HomeAssistant and 1 CT node for Frigate container. When I do Proxmox node reboot, it shuts down all containers and then hangs, nothing happens and it does not start up. It becomes unavailable on my router and web UI goes down also. Need to do complete NUC reboot over power switch, then everything is starting up normally.
I can do shutdown/reboot of single nodes inside Proxmox without any problem, my both nodes are shutting down/rebooting normally. Issue happens only if I reboot complete Proxmox node.
My Proxmox is fully up to date:
At first I was having "watchdog did not stop" message at the end of my logs.
I have updated system.conf with:
RuntimeWatchdogSec=0
ShutdownWatchdogSec=0
So now I dont see "watchdog did not stop" message anymore, but nothing has changed in terms of reboot hang.
Inside Proxmox node I have 1 VM node for HomeAssistant and 1 CT node for Frigate container. When I do Proxmox node reboot, it shuts down all containers and then hangs, nothing happens and it does not start up. It becomes unavailable on my router and web UI goes down also. Need to do complete NUC reboot over power switch, then everything is starting up normally.
I can do shutdown/reboot of single nodes inside Proxmox without any problem, my both nodes are shutting down/rebooting normally. Issue happens only if I reboot complete Proxmox node.
My Proxmox is fully up to date:
Code:
pveversion -v
proxmox-ve: 7.1-1 (running kernel: 5.13.19-4-pve)
pve-manager: 7.1-10 (running version: 7.1-10/6ddebafe)
pve-kernel-helper: 7.1-9
pve-kernel-5.13: 7.1-7
pve-kernel-5.11: 7.0-10
pve-kernel-5.13.19-4-pve: 5.13.19-8
pve-kernel-5.13.19-3-pve: 5.13.19-7
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.11.22-4-pve: 5.11.22-9
ceph-fuse: 15.2.14-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.1
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-6
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.1-2
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.1-1
libpve-storage-perl: 7.0-15
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.3.0-1
proxmox-backup-client: 2.1.5-1
proxmox-backup-file-restore: 2.1.5-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-5
pve-cluster: 7.1-3
pve-container: 4.1-3
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-4
pve-ha-manager: 3.3-3
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.1-1
pve-xtermjs: 4.16.0-1
qemu-server: 7.1-4
smartmontools: 7.2-1
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.2-pve1At first I was having "watchdog did not stop" message at the end of my logs.
Code:
Feb 05 10:39:59 proxmox systemd[1]: dm-event.service: Succeeded.
Feb 05 10:39:59 proxmox systemd[1]: Stopped Device-mapper event daemon.
Feb 05 10:39:59 proxmox systemd[1]: Reached target Shutdown.
Feb 05 10:39:59 proxmox systemd[1]: Reached target Final Step.
Feb 05 10:39:59 proxmox systemd[1]: systemd-reboot.service: Succeeded.
Feb 05 10:39:59 proxmox systemd[1]: Finished Reboot.
Feb 05 10:39:59 proxmox systemd[1]: Reached target Reboot.
Feb 05 10:39:59 proxmox systemd[1]: Shutting down.
Feb 05 10:39:59 proxmox systemd[1]: Using hardware watchdog 'Software Watchdog', version 0, device /dev/watchdog
Feb 05 10:39:59 proxmox systemd[1]: Set hardware watchdog to 10min.
Feb 05 10:39:59 proxmox kernel: watchdog: watchdog0: watchdog did not stop!
Feb 05 10:39:59 proxmox systemd-shutdown: Syncing filesystems and block devices.
Feb 05 10:39:59 proxmox systemd-shutdown: Sending SIGTERM to remaining processes...I have updated system.conf with:
RuntimeWatchdogSec=0
ShutdownWatchdogSec=0
So now I dont see "watchdog did not stop" message anymore, but nothing has changed in terms of reboot hang.
Code:
Feb 05 11:43:34 proxmox systemd[1]: lvm2-monitor.service: Succeeded.
Feb 05 11:43:34 proxmox systemd[1]: Stopped Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling.
Feb 05 11:43:34 proxmox dmeventd[418]: dmeventd detected break while being idle for 0 second(s), exiting.
Feb 05 11:43:34 proxmox dmeventd[418]: dmeventd shutting down.
Feb 05 11:43:34 proxmox systemd[1]: Stopping Device-mapper event daemon...
Feb 05 11:43:34 proxmox systemd[1]: dm-event.service: Succeeded.
Feb 05 11:43:34 proxmox systemd[1]: Stopped Device-mapper event daemon.
Feb 05 11:43:34 proxmox systemd[1]: Reached target Shutdown.
Feb 05 11:43:34 proxmox systemd[1]: Reached target Final Step.
Feb 05 11:43:34 proxmox systemd[1]: systemd-reboot.service: Succeeded.
Feb 05 11:43:34 proxmox systemd[1]: Finished Reboot.
Feb 05 11:43:34 proxmox systemd[1]: Reached target Reboot.
Feb 05 11:43:34 proxmox systemd[1]: Shutting down.
Feb 05 11:43:34 proxmox systemd-shutdown: Syncing filesystems and block devices.
Feb 05 11:43:35 proxmox systemd-journald[408]: Journal stopped