HACK - KVM only - Control order and delay starting VMs in proxmox

[xagaba]

Hi, because I need to control the start order of VMs and set a delay between them in some circunstances, I write a little bash script to accomplish this with KVM vms in proxmox.

/usr/sbin/qmcontrol

#!/bin/bash
#
# qmcontrol v0.03
#
# - changed Starting VM Message to something more meaningful (zextra comments)
# - added DEFLAY (default delay for VMs without delay)
# - check only for VMid.conf files (udo suggestion)
#

DEFLAY=5
PROG=/usr/sbin/qm

if [[ -z "$1" || "$1" != "startall" && "$1" != "stopall" ]]; then
    echo "Usage: qmcontrol <startall>|<stopall>"
    exit
fi

for file in `ls /etc/qemu-server/ | grep conf$`
do
    if [ 1 -eq `cat /etc/qemu-server/$file | grep onboot: | awk '{print $2}'` ]; then
	vmboot="$vmboot $file"
    fi
done

for file in $vmboot
do
    order=`cat /etc/qemu-server/$file | grep order: | awk '{print $2}'`
    if [ -z "$order" ]; then
	order="9999"
    fi
    vmsort="$vmsort $order $file"
done

if [ "$1" = "startall" ]; then
    vmselect=`echo ${vmsort//.conf/.} | tr "." "\n" | sort -bg`
else
    vmselect=`echo ${vmsort//.conf/.} | tr "." "\n" | sort -bgr`
fi

count=0
for vmid in $vmselect
do
    if [ `expr $count % 2` -eq 1 ]; then
	if [ "$1" = "startall" ]; then
            delay=`cat /etc/qemu-server/$vmid.conf | grep delay: | awk '{print $2}'`
            if [ -z $delay ]; then
                delay=$DEFLAY
            fi
	    echo "Starting VM $vmid, waiting $delay seconds to allow VM $vmid services to start"
	    $PROG start $vmid 2>/dev/null
            sleep $delay
	else
	    echo "Stopping VM $vmid"
	    $PROG shutdown $vmid 2>/dev/null
	    $PROG wait $vmid 2>/dev/null
	fi
    fi
    ((count++))
done

if [ "$1" = "stopall" ]; then
    echo "Stopping any other VMs"
    $PROG stopall 2>/dev/null
fi

The script replace the /usr/sbin/qm startall and stopall commands to introduce two new settings in the vm configuration, order: and delay:

You need to put this settings manually inside the configuration files you find at /etc/qemu-server.

ostype: other
memory: 2048
sockets: 1
name: nas
ide2: none,media=cdrom
vlan0: virtio=42:00:66:00:4C:00
bootdisk: virtio0
boot: cd
freeze: 0
cpuunits: 1000
acpi: 1
kvm: 1
virtio0: local:101/vm-101-disk-1.raw
onboot: 1
cores: 1
order: 2
delay: 30

If you miss any of them or both, the script assigns the latest order to this vms and set his delay to DEFLAY value in script.

To activate the script you need to edit the /etc/init.d/qemu-server file and change the variable PROG to use qmcontrol instead of qm

Need to look like this

...
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
PROG=/usr/sbin/qmcontrol
DESC="Qemu Server"
...

I'm a newbie bash scripter, so any advice or modification will be appreciated !

 

[cks]

Wonderful script !

A suggestion for next version : use a default delay taken from some global configuration file (not sure that /etc/vzdump.conf is the right place ?) instead of 0

I hope your script will enter one of the next official proxmox iso btw (and to see soon the 2 new fields to set values from web UI also)

Great job xagaba !

 

[Ninjix]

Very nice. Thanks for sharing!

 

[macday]

A big thank you from me too

 

[udo]

...

...
for file in `ls /etc/qemu-server/`
...

...
I'm a newbie bash scripter, so any advice or modification will be appreciated !

Hi,
perhaps you should look only for ".conf" files - in your case you use also "101.conf.save", which perhaps someone use as reference copy?
Like

ls /etc/qemu-server/ | grep  conf$

Udo

 

[xagaba]

cks - This is only a little hack before we get the new 2.0 version from the proxmox guys. For sure that they have this features implemented or planned. I put a DEFLAY variable in the header. Thanks for your support !

udo- Thanks for your suggestion, It's updated in the script right now.

 

[cks]

Thanks to you for the script xagaba !

 

[nicolasdiogo]

T H A N K S

it is a really good idea and very simple to implement as well.


Many thanks,

Nicolas

 

[nicolasdiogo]

just a quick check i get this message when using your scripts:

/usr/sbin/qmcontrol: line 19: [: 1: unary operator expected

this one is the line in question: /usr/sbin/qmcontrol: line 19: [: 1: unary operator expected should i be worried?

 

[xagaba]

Sorry for the delay in reply nicolasdiogo ...

Are you using the KVM virtualization or the OpenVZ ?
There are any files in your /etc/qemu-server directory ?
Can you show me how you put the order: and delay: atributes in the vm files ?
There are almost one vm with the onboot: atribute active ?

 

[zextra]

xagba,

first, thanks for this excellent script!

However.. In my opinion, lines 2 and 3 in following code block should be swapped:

echo "Starting VM $vmid with $delay seconds delay"
$PROG start $vmid 2>/dev/null
sleep $delay

..so it looks like this:

echo "Starting VM $vmid with $delay seconds delay"
sleep $delay
$PROG start $vmid 2>/dev/null

I said "in my opinion", because configuring delays is a bit weird without that change. Currently, it works like this (just outline):

1. find all config files
2. sort them by their order value
3. start first ordered VM
4. execute delay
5. repeat 3, until no more VMs to start.

Now, steps 3 and 4 are what feels weird. In order to set specific delay on vm 105, I have to find and edit config file of VM which runs prior to vm 105. There might be 1-2 files to search, but there just as well might be 30 other files...

So, if you swap thoes two lines, it becomes more logical - you configure boot order AND delay that VM should execute prior to starting itself. Also, console output mirrors this kind of behavior: it says "Starting VM 102 with 45 seconds delay" and then it just sits there, waiting for 45 secs to pass, before launching VM 102. Without this change, it says "Starting VM 102 with 45 seconds delay" at which point VM102 is already running, but 45 secs delay is executed anyway, before proceeding to next VM.

 

[xagaba]

Thanks zextra for your comments !

I make this script to prevent certain vm's to start before some services from other vm's are ready to be used.

This is the reason to put the delay after starting the vmid to allow for his services to be available to others before starting new vm's (we need to guess how much seconds we need for this to happend)

The message, as you points correctly, it's wrong or bogus and need to be something like this:

echo "Starting VM $vmid, waiting $delay seconds to allow VM $vmid services to start"

I hope that this clarification agrees with your thoughts.

P.D. Changed message in script

 

[nicolasdiogo]

Sorry for the delay in reply nicolasdiogo ...

Are you using the KVM virtualization or the OpenVZ ?
There are any files in your /etc/qemu-server directory ?
Can you show me how you put the order: and delay: atributes in the vm files ?
There are almost one vm with the onboot: atribute active ?

i am getting error on line:

if [ 1 -eq `cat /etc/qemu-server/$file | grep onboot: | awk '{print $2}'` ]; then

i would presume it to be something to do with the fact that one of the conf files does not have the onboot:

maybe?

thanks,

 

[findfast]

Maybe in next release will be added option for delays in seconds from initial start of system
for every VMs only if checked box to enable it?

More complicated dependences could be added in 2.x or perfomed by scripts as above.