Shutting down all VM / CT in parallel

[DynFi User]

I would like to know if you know how I can shutdown ALL VM / CT in parallel ? (as in the GUI).
I have setup a script to shutdown VM, CT and then set the noout flag on the CEPH Cluster and finally shutdown the server.

Problem is that the script iterates with a for loop (which causes VM to be shutdown one by one) and not in parallel (which should greatly reduce the shutdown time.

Here is my script :

#!/bin/bash

VM_LIST=/tmp/list_VM.list
CT_LIST=/tmp/list_CT.list

# We catch the VM & CT from each server
qm list | awk 'NR>1{ print $1 }' > $VM_LIST
pct list | awk 'NR>1{ print $1 }' > $CT_LIST

mapfile -t listvm < $VM_LIST

for i in ${listvm[@]};
  do
    echo "I will shutdown VM $i"
    qm shutdown $i
    echo "VM $i has been shutdown"
    echo ""
  done


mapfile -t listct < $CT_LIST

for i in ${listct[@]};
  do
        echo "I will shutdown CT $i"
        pct shutdown $i
    echo "CT $i has been shutdown"
    echo ""
  done

echo "We set the noout flag on CEPH Cluster:"
ceph osd set noout

echo "We now can safely shutdown the server"
shutdown -h now


echo "End of script and deletion of VM / CT lists"
rm $VM_LIST
rm $CT_LIST

 

[bbgeek17]

The closest you can get to is via https://pve.proxmox.com/pve-docs/api-viewer/#/nodes/{node}/stopall

Even with the above API the stopping will be done one at a time, as a "configuration lock" is taken on each VM/CT shutdown.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[PeteMox]

I would like to know if you know how I can shutdown ALL VM / CT in parallel ? (as in the GUI).
I have setup a script to shutdown VM, CT and then set the noout flag on the CEPH Cluster and finally shutdown the server.

Problem is that the script iterates with a for loop (which causes VM to be shutdown one by one) and not in parallel (which should greatly reduce the shutdown time.

Here is my script :

#!/bin/bash

VM_LIST=/tmp/list_VM.list
CT_LIST=/tmp/list_CT.list

# We catch the VM & CT from each server
qm list | awk 'NR>1{ print $1 }' > $VM_LIST
pct list | awk 'NR>1{ print $1 }' > $CT_LIST

mapfile -t listvm < $VM_LIST

for i in ${listvm[@]};
  do
    echo "I will shutdown VM $i"
    qm shutdown $i
    echo "VM $i has been shutdown"
    echo ""
  done


mapfile -t listct < $CT_LIST

for i in ${listct[@]};
  do
        echo "I will shutdown CT $i"
        pct shutdown $i
    echo "CT $i has been shutdown"
    echo ""
  done

echo "We set the noout flag on CEPH Cluster:"
ceph osd set noout

echo "We now can safely shutdown the server"
shutdown -h now


echo "End of script and deletion of VM / CT lists"
rm $VM_LIST
rm $CT_LIST

This is exactly what I was looking for, thanks a lot!

 

[Kingneutron]

You are already writing the VM and CT running list to a file, pass that on to GNU parallel (or possibly xargs)
Note the ' grep running ' so you skip any stopped
For hibernation, substitute ' qm suspend $vmid --todisk '

# BEGIN bash
# 1 2 3
# processor : 7
lastcpu=$(grep processor /proc/cpuinfo |tail -n 1 |awk '{print $3}')
[ $lastcpu -gt 1 ] && let lastcpu=$lastcpu-1

[ -e "$VM_LIST" ] && rm -f "$VM_LIST" # delete the old file ONLY IF it exists
for vmid in $(qm list |grep running |awk '{print $1}'); do
echo "qm shutdown $vmid" >>"$VM_LIST"
done

cat "$VM_LIST" |parallel -j $lastcpu --progress

 

[Kingneutron]

You are already writing the VM and CT running list to a file, pass that on to GNU parallel (or possibly xargs)
Note the ' grep running ' so you skip any stopped
For hibernation, substitute ' qm suspend $vmid --todisk '

# BEGIN bash
# 1 2 3
# processor : 7
lastcpu=$(grep processor /proc/cpuinfo |tail -n 1 |awk '{print $3}')
[ $lastcpu -gt 1 ] && let lastcpu=$lastcpu-1

[ -e "$VM_LIST" ] && rm -f "$VM_LIST" # delete the old file ONLY IF it exists
for vmid in $(qm list |grep running |awk '{print $1}'); do
echo "qm shutdown $vmid" >>"$VM_LIST"
done

cat "$VM_LIST" |parallel -j $lastcpu --progress

Note - if you elect to Hibernate all running VMs in parallel, this will generate considerable disk I/O load. Best if you have mirrored or raidz2 if zfs, or fast nvme.

I managed to hibernate 5x running VMs to several backing-storage disks (spinning hd + 2x nvme) in ~1M+14S testing this script, which should be adequate for UPS. YMMV