Testing iSCSI and HA Proxmox VE 3.1 with vmware fusion

[muetst]

To learn more about Proxmox I built a cluster PVE 3.1 on my Macbook Pro with two Proxmox servers and a NFS Freenas for storage.
It works great and I learning how Proxmox is working. I even can start a Centos 32 bits machine on this cluster. I can do a migrate, open a console to the Centos guest and make snapshots...all very cool.
Some parts I do not understand about Proxmox. Working with vSphere, Hyperv, XEN server or RHEV, I can create a filesystem on the iSCSI LUN mounted to the hypervisors. With Proxmox I only can create a LVM but there is no support for snapshots using the Proxmox GUI (compare to VMware I create a VMFS on the ISCSI LUN and then put my VMDK's on the VMFS and make snapshots) I miss this option in Proxmox, Is it so difficult to create a filesystem on top of a iSCSI LUN? For now I skipped the iSCSI config and using NFS.

Second I want to test is HA, but i need a fence device and my VMware fusion does not have a option for a fence device. Is there a way to fool the proxmox cluster so I can test HA and see how it is working? You can stop and start VM's with CLI command vmrun.

At work we are testing Oracle VM, but we mis the option for a good backup and snapshot. I only use Oracle VM because of the licensing of Oracle.
as a virtualization specialist I am very impressed about Proxmox! maybe one day I become a PCP (I am a VCP for VMware).

 

[muetst]

I find this interesting script: http://meinit.nl/nl/virtualbox-fencing-and-red-hat-enterprise-linux-cluster-suite
Now I have to find a way to make a script using VMware Fusion.

 

[m.ardito]

here http://en.wikipedia.org/wiki/VMware_VMFS it says that an alternative is NFS.
So, you if you need snapshots, you can use qcow2 on NFS (or local) or ceph ( see http://pve.proxmox.com/wiki/Live_Snapshots) in pve...

Marco

 

[muetst]

here http://en.wikipedia.org/wiki/VMware_VMFS it says that an alternative is NFS.
So, you if you need snapshots, you can use qcow2 on NFS (or local) or ceph ( see http://pve.proxmox.com/wiki/Live_Snapshots) in pve...

Marco

Yep thats what I already know, my question is why Proxmox does not put a file system on top of the VLM so you can use snapshots.

Oke, now I am testing fencing. This is my fence script put into /usr/sbin name fence_fusion

#!/bin/sh -x

# Script to stop and start a virtual machine.
# The only required argument is machinename.

eval $(cat -)

# I use Apple Mac OS X, but any OS may be used.
vmrun="/Applications/VMware\ Fusion.app/Contents/Library/vmrun"

usage () {
/bin/echo "Usage: $0 -a NAME [-o ACTION]"
/bin/echo
/bin/echo " -a NAME"
/bin/echo " The name of the virtual machine to be fenced."
/bin/echo " In case it contains spaces, use double quotes."
/bin/echo " -o ACTION"
/bin/echo " What to do; start|stop|reboot(default)."
/bin/echo
exit 1
}

while [ "$#" -gt 0 ] ; do
case "$1" in
-a)
if [ "$2" ] ; then
vm="$2"
shift ; shift
else
/bin/echo "Missing value for $1."
/bin/echo
usage
shift
fi
;;
-o)
if [ "$2" ] ; then
action="$2"
shift ; shift
else
/bin/echo "Missing value for $1."
/bin/echo
usage
shift
fi
;;
*)
/bin/echo "Not a know option, $1."
usage
shift
;;
esac
done

if [ ! "${action}" ] ; then
action=reboot
fi

if [ ! "${vm}" ] ; then
/bin/echo "Error, please specify a name."
usage
fi

check() {
ssh loginname@192.168.0.108 "$vmrun list" > /dev/null 2>&1
if [ ${?} != 0 ] ; then
/bin/echo "Error, VM ${vm} not found, choose one of these:"
ssh loginname@192.168.0.108 "$vmrun list"
exit 1
fi
}


stop() {
ssh loginname@192.168.0.108 "$vmrun stop /Volumes/LaCie/VM/${vm}.vmwarevm/${vm}.vmx > /dev/null 2>&1"
}

start() {
ssh loginname@192.168.0.108 "$vmrun start /Volumes/LaCie/VM/${vm}.vmwarevm/${vm}.vmx > /dev/null 2>&1"
}

reboot() {
stop
sleep 3
start
}

case $action in
start)
check
start
;;
stop)
check
stop
;;
reboot)
check
reboot
;;
*)
/bin/echo "Unknown action: $action"
/bin/echo
usage
;;
esac

My cluster.conf.new looks like:

<?xml version="1.0"?>
<cluster config_version="40" name="pmcluster">
<cman keyfile="/var/lib/pve-cluster/corosync.authkey" two_node="1" expected_votes="1"/>
<fencedevices>
<fencedevice agent="fence_fusion" name="fusion"/>
</fencedevices>
<clusternodes>
<clusternode name="proxmox1" nodeid="1" votes="1">
<fence>
<method name="1">
<device name="fusion" vm="proxmox1"/>
</method>
</fence>
</clusternode>
<clusternode name="proxmox2" nodeid="2" votes="1">
<fence>
<method name="1">
<device name="fusion" vm="proxmox2"/>
</method>
</fence>
</clusternode>
</clusternodes>
<rm>
<pvevm autostart="1" vmid="100"/>
</rm>
</cluster>

When I try to activate my config I get config validation failed: unknown error (500)
Something is configured wrong, but I can not find the problem. The fence_fusion is exactable.

when I run ./fence_fusion -a proxmox2 -o reboot the server does a reboot. (I must disable the $(cat -) option when I' am testing).

 

[muetst]

Extra info

running command: ccs_config_validate -l /etc/pve/cluster.conf.new

Relax-NG validity error : Extra element fence in interleave
tempfile:7: element clusternodes: Relax-NG validity error : Element clusternode failed to validate content
tempfile:8: element clusternode: Relax-NG validity error : Element clusternodes has extra content: clusternode
Configuration fails to validate

But if i do
root@proxmox1:/etc/pve# fence_node proxmox2
fence proxmox2 success
root@proxmox1:/etc/pve# fence_node proxmox1
fence proxmox1 success
root@proxmox1:/etc/pve#

 

[dietmar]

Attribute 'vm' is not allowed in device tag.(seems you skipped the first warning)

tempfile:11: element device: Relax-NG validity error : Invalid attribute vm for element device

The xml verification fail, because there are no definitions for you self made agent in the relax-ng schema.

 

[dietmar]

To generate a correct xml cluster schema, your agent must be able to provide some metadata.
For example:

# fence_apc -o metadata

The ccs_update_schema tool used that information to update /usr/share/cluster/cluster.rng

 

[muetst]

To generate a correct xml cluster schema, your agent must be able to provide some metadata.
For example:

# fence_apc -o metadata

The ccs_update_schema tool used that information to update /usr/share/cluster/cluster.rng

Thanks for the answer. I have discovered that the Attribute 'vm' is not allowed in device tag and I try to replace it with option "nodename"
<fencedevices>
<fencedevice agent="fence_fusion" name="fusion1" nodename="proxmox1"/>
<fencedevice agent="fence_fusion" name="fusion2" nodename="proxmox2"/>
</fencedevices>
This is accepted by the relax-ng schema, but as aspected does not do the trick to reboot the virtual failed cluster node. The VM running in my Proxmox cluster does fail over to the other node when I virtual unplug the network adapter, but the fence_fusion script does not work (this is because my script does not give the cluster any metadata. I changed my script a little in the hope it does work:

ssh username@192.168.0.108 "$vmrun stop /Volumes/LaCie/VM/${nodename}.vmwarevm/${nodename}.vmx > /dev/null 2>&1"
I replaced the "vm" with "nodename". Is there a chance proxmox is making a fence_ script for virtual box or vmware fusion/workstation to build a test/learning cluster on top of a laptop or desktop computer? Both VMware and Oracle virtualbox support reboot with a cli. It will be a nice option for people like me who want to learn more about new virtualization options.