VM shutdown on host shutdown.
- Thread starter Hyien
- Start date
this is not standard. You need to configure this.
In each VM under "Options" you can specify two things:
- Start at boot
- Start/Shutdown order
The order takes a numeric values and is handled forward (1, 2, 3, 4 etc.) on start and on reverse (10, 9, 8, ...) on shutdown. Of course you can put one number on multiple VMs to shutdown them in parallel.
Also make sure to enable ACPI support so the VM can be powered off.
In each VM under "Options" you can specify two things:
- Start at boot
- Start/Shutdown order
The order takes a numeric values and is handled forward (1, 2, 3, 4 etc.) on start and on reverse (10, 9, 8, ...) on shutdown. Of course you can put one number on multiple VMs to shutdown them in parallel.
Also make sure to enable ACPI support so the VM can be powered off.
the guests will be powered off in any case (if possible), the startup/.. settings just gives you more control over the order.
@fabian is this still the preferred method to shutdown guests without shutting down host?
# service pve-manager stop
I am trying to coordinate safe shutdown of Ceph but I need it to shutdown 1) guests 2) osds and then 3) host. Unless I am mistaken and there is a simple way to gracefully shutdown everything automatically. I have not come across anything about safe shutdown an entire Ceph cluster except for noout individual osds for maintenance.
# service pve-manager stop
I am trying to coordinate safe shutdown of Ceph but I need it to shutdown 1) guests 2) osds and then 3) host. Unless I am mistaken and there is a simple way to gracefully shutdown everything automatically. I have not come across anything about safe shutdown an entire Ceph cluster except for noout individual osds for maintenance.
Last edited:
To shut down VMs the following script sample might help.
I did not test it and just have thrown it together
HTH
I did not test it and just have thrown it together
Bash:
#!/bin/bash
#option="stop"
option="shutdown"
for vm in `sudo qm list | awk '{print $1}' | grep -v VMID`
do
echo "$option VM with ID: $vm"
command="sudo qm $vm $option"
echo "--> command: $command"
# uncomment the following line to execute the command.
#sudo $command
echo " "
doneHTH
Thanks so much for taking the time to do this. I'll definitely give it a try. I spent some time digging around Prox and found a handful of files related to 'pve-manager' but no luck finding the exact scripting. It's called as a service but it's actually not a daemon, so I was hoping to find the script. I'll try both and see how it works.To shut down VMs the following script sample might help.
I did not test it and just have thrown it together
Bash:#!/bin/bash #option="stop" option="shutdown" for vm in `sudo qm list | awk '{print $1}' | grep -v VMID` do echo "$option VM with ID: $vm" command="sudo qm $vm $option" echo "--> command: $command" # uncomment the following line to execute the command. #sudo $command echo " " done
HTH
there is a pve-guests service (used to be called pve-manager and still aliased) that is not a daemon, but a oneshot service called at startup (start all guests configured to start on boot) and on shutdown (shutdown all running guests):
you can see the commands that get called on shutdown (ExecStop) - if you run them manually you might want to drop the '--no-output' though
Code:
$ systemctl cat pve-guests.service
# /lib/systemd/system/pve-guests.service
[Unit]
Description=PVE guests
ConditionPathExists=/usr/bin/pvesh
RefuseManualStart=true
RefuseManualStop=true
Wants=pvestatd.service
Wants=pveproxy.service
Wants=spiceproxy.service
Wants=pve-firewall.service
Wants=lxc.service
After=pveproxy.service
After=pvestatd.service
After=spiceproxy.service
After=pve-firewall.service
After=lxc.service
After=pve-ha-crm.service pve-ha-lrm.service
[Service]
Environment="PVE_LOG_ID=pve-guests"
ExecStartPre=-/usr/share/pve-manager/helpers/pve-startall-delay
ExecStart=/usr/bin/pvesh --nooutput create /nodes/localhost/startall
ExecStop=-/usr/bin/vzdump -stop
ExecStop=/usr/bin/pvesh --nooutput create /nodes/localhost/stopall
Type=oneshot
RemainAfterExit=yes
TimeoutSec=infinity
[Install]
WantedBy=multi-user.target
Alias=pve-manager.serviceyou can see the commands that get called on shutdown (ExecStop) - if you run them manually you might want to drop the '--no-output' though
Thanks @fabian - that's exactly what I was hoping to find. Much appreciatedthere is a pve-guests service (used to be called pve-manager and still aliased) that is not a daemon, but a oneshot service called at startup (start all guests configured to start on boot) and on shutdown (shutdown all running guests):
Code:$ systemctl cat pve-guests.service # /lib/systemd/system/pve-guests.service [Unit] Description=PVE guests ConditionPathExists=/usr/bin/pvesh RefuseManualStart=true RefuseManualStop=true Wants=pvestatd.service Wants=pveproxy.service Wants=spiceproxy.service Wants=pve-firewall.service Wants=lxc.service After=pveproxy.service After=pvestatd.service After=spiceproxy.service After=pve-firewall.service After=lxc.service After=pve-ha-crm.service pve-ha-lrm.service [Service] Environment="PVE_LOG_ID=pve-guests" ExecStartPre=-/usr/share/pve-manager/helpers/pve-startall-delay ExecStart=/usr/bin/pvesh --nooutput create /nodes/localhost/startall ExecStop=-/usr/bin/vzdump -stop ExecStop=/usr/bin/pvesh --nooutput create /nodes/localhost/stopall Type=oneshot RemainAfterExit=yes TimeoutSec=infinity [Install] WantedBy=multi-user.target Alias=pve-manager.service
you can see the commands that get called on shutdown (ExecStop) - if you run them manually you might want to drop the '--no-output' though
@fabian - Finally getting a chance to try out your suggestion...
Is there an ALL value that can be passed to
pvesh create /nodes/{host}/stopall works except for HA-enabled VMs.Is there an ALL value that can be passed to
ha-manager set <sid> -state started/stopped ? If not, I can at least manually stop each HA member in my script.
Last edited:
HA resources are handled when the HA services are stopped on shutdown (and what happens then depends on the configured policies).. IIRC there is some logic there to detect the system is actually shutting down (as opposed to just the service stopping as part of a package upgrade), so probably your safest bet is to loop over the configured resources and set them to stopped (and then wait for the state to settle).
Is there a global parameter that sets the amount of time PVE host waits on for the Guests to properly shutdown or do we have to set that amount of time for each vm in the startup/shutdown properties of the vm?
Does PVE shutdown VMs without startup/shutdown priority at last?
Does PVE shutdown VMs without startup/shutdown priority at last?
There is a 180 second default per VM.
It starts them last and shuts them down first. See the manual for more details.
So there’s no way to change the global default and I have to increase it per-VM if necessary. Thank you