Feature request: Hibernate VMs on PVE host reboot

The #17 solution is not perfect, I have multiple VMs, Some VMs hibernate successfully, but some don't. It seems that the script on #17 and PVE shutdown process both execute in parallel.

So I use custom command to do this manually.
I created a script file reboot.sh

Code:
#!/bin/bash
qm suspend 102 --todisk
qm suspend 101 --todisk
qm guest cmd 100 "suspend-disk"
reboot

Everytime I want to reboot, I execute this script in PVE console.

The VM 100 is windows operating system with QEMU Guest Agent installed, using "guest cmd" to hibernate is a better way.
 
Last edited:
The answer in #17 creates a parallel job and relies on timeouts, this will not work reliably. A proper way of doing this would be to:

1. Create a script e.g. /var/lib/vz/snippets/pve-hibernate-all:
Bash:
#!/bin/bash

qm list | grep running | awk -F'[^0-9]*' '$0=$2' | while read -r vm_id; do qm suspend $vm_id --todisk 1; done;

2. chmod 755 /var/lib/vz/snippets/pve-hibernate-all
3. Edit /etc/systemd/system/pve-manager.service and add the the following line:
Bash:
ExecStop=/var/lib/vz/snippets/pve-hibernate-all

Such that in context it reads like:
Bash:
[Service]
Environment="PVE_LOG_ID=pve-guests"
ExecStartPre=-/usr/share/pve-manager/helpers/pve-startall-delay
ExecStart=/usr/bin/pvesh --nooutput create /nodes/localhost/startall
ExecStop=/var/lib/vz/snippets/pve-hibernate-all
ExecStop=-/usr/bin/vzdump -stop
ExecStop=/usr/bin/pvesh --nooutput create /nodes/localhost/stopall

(The order is important here.)

Then reload systemd cache (systemctl daemon-reload) or simply reboot to pick up the changes.

After this, all running VMs will be hibernated when PVE stops regardless of how you do it. It works nicely with autostart on boot as well.
 
  • Like
Reactions: omcom and leeyoo
The answer in #17 creates a parallel job and relies on timeouts, this will not work reliably. A proper way of doing this would be to:

1. Create a script e.g. /var/lib/vz/snippets/pve-hibernate-all:
Bash:
#!/bin/bash

qm list | grep running | awk -F'[^0-9]*' '$0=$2' | while read -r vm_id; do qm suspend $vm_id --todisk 1; done;

2. chmod 755 /var/lib/vz/snippets/pve-hibernate-all
3. Edit /etc/systemd/system/pve-manager.service and add the the following line:
Bash:
ExecStop=/var/lib/vz/snippets/pve-hibernate-all

Such that in context it reads like:
Bash:
[Service]
Environment="PVE_LOG_ID=pve-guests"
ExecStartPre=-/usr/share/pve-manager/helpers/pve-startall-delay
ExecStart=/usr/bin/pvesh --nooutput create /nodes/localhost/startall
ExecStop=/var/lib/vz/snippets/pve-hibernate-all
ExecStop=-/usr/bin/vzdump -stop
ExecStop=/usr/bin/pvesh --nooutput create /nodes/localhost/stopall

(The order is important here.)

Then reload systemd cache (systemctl daemon-reload) or simply reboot to pick up the changes.

After this, all running VMs will be hibernated when PVE stops regardless of how you do it. It works nicely with autostart on boot as well.
nice but not working for my 8.0.4.
by your idea i change the plug point to poweroff and reboot command

change original poweroff and reboot

Bash:
mv /usr/sbin/reboot /usr/sbin/reboot_now
mv /usr/sbin/poweroff /usr/sbin/poweroff_now


and eidt bit your code with waiting
Bash:
cat > /var/lib/vz/snippets/pve-hibernate-all <<EOF
qm list | grep running | awk -F '[^0-9]*' '\$0=\$2' | while read -r vm_id; do qm suspend \$vm_id --todisk 1; done;
while qm list | grep running>/dev/null; do sleep 1; done;
EOF
then rebuild poweroff and reboot with permissions
Bash:
cat > /usr/sbin/poweroff <<EOF
/var/lib/vz/snippets/pve-hibernate-all
/usr/sbin/poweroff_now
EOF
cat > /usr/sbin/reboot <<EOF
/var/lib/vz/snippets/pve-hibernate-all
/usr/sbin/reboot_now
EOF
chmod 777 /usr/sbin/reboot  /usr/sbin/poweroff
 
Last edited:
  • Like
Reactions: fireon
i am new to linux and proxmox, but
Code:
pvesh create /nodes/localhost/suspendall
seems to work fine

example
Code:
/etc/systemd/system/pve-guests.service
Bash:
[Service]
Environment="PVE_LOG_ID=pve-guests"
ExecStartPre=-/usr/share/pve-manager/helpers/pve-startall-delay
ExecStart=/usr/bin/pvesh --nooutput create /nodes/localhost/startall
ExecStop=-/usr/bin/vzdump -stop
ExecStop=/usr/bin/pvesh --nooutput create /nodes/localhost/suspendall
ExecStop=/usr/bin/pvesh --nooutput create /nodes/localhost/stopall
Type=oneshot
RemainAfterExit=yes
TimeoutSec=infinity
 
  • Like
Reactions: fireon
adding complete code for /etc/systemd/system/pve-guests.service and commands for Newbies (like me)

[Unit]
Description=Hibernate all VM
After=multi-user.target

[Service]
Environment="PVE_LOG_ID=pve-guests"
ExecStartPre=-/usr/share/pve-manager/helpers/pve-startall-delay
ExecStart=/usr/bin/pvesh --nooutput create /nodes/localhost/startall
ExecStop=-/usr/bin/vzdump -stop
ExecStop=/usr/bin/pvesh --nooutput create /nodes/localhost/suspendall
ExecStop=/usr/bin/pvesh --nooutput create /nodes/localhost/stopall
Type=oneshot
RemainAfterExit=yes
TimeoutSec=infinity

[Install]
WantedBy=multi-user.target

save the file
Next step would be to enable the service

sudo systemctl enable pve-guests.service


to stop use below
sudo systemctl stop pve-guests.service

to Disable
sudo systemctl disable pve-guests.service
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!