Understanding cluster.conf

[ksosez]

Hey there,

I am trying to understand cluster.conf and its relationship to corosync.

It appears from here I can do totem settings in the cluster.conf, can I do more advanced things to corosync such as using unicast as referenced here and in the /usr/share/doc/corosync-pve/examples/corosync.conf.example.udpu


I would really like to turn on the udpu/unicast option since Singlehop does not support unicast and I would like to manage all the systems from one interface.

 

[e100]

I am the one who wrote this: http://pve.proxmox.com/wiki/Multicast_notes#Changing_netmtu

According to the docs I have read yes you can configure lots of stuff in the cluster config, not just totem.
I also made the note on the wiki that I have not tested that yet.

man 5 cman

The info here looks promising:
https://bugzilla.redhat.com/show_bug.cgi?id=657041

Notes from that page:
1. To enable the UDPU transport, manually add the line <cman transport="udpu"/> to the cluster.conf file:

<cman [B]transport="udpu"[/B] keyfile="/var/lib/pve-cluster/corosync.authkey">

2. You can not have a <totem> tag in your cluster.conf if you also want udpu.
3. You must restart the whole cluster after making the change

One last note, the issue with the totem tag and udpu issue was recently fixed, maybe Proxmox has the fix already?
https://bugzilla.redhat.com/show_bug.cgi?id=695795

I tested the change to my cluster.conf and it does seem to work:
/var/log/cluster/corosync.log:

Feb 01 22:19:05 corosync [TOTEM ] Initializing transport (UDP/IP [B]Unicast[/B]).

 

[ksosez]

Thanks a lot e100,

Maybe we should add this to the wiki?

I still need to figure out how to specify the other IP's of the other systems.

 

[e100]

My understanding is that udpu uses broadcast to communicate so it might not be necessary to specify the other addresses.

Did you get this working and get proper cluster communication going?

 

[dietmar]

My understanding is that udpu uses broadcast to communicate so it might not be necessary to specify the other addresses.

AFAIK udpu only use unicast. See 'man corosync.conf':

              To avoid the use of  multicast  entirely,  a  unicast  transport
              parameter "udpu" can be specified.  This requires specifying the
              list of members that could potentially make  up  the  membership
              before deployment.

 

[ksosez]

I havent had a chance to setup a test environment yet. Hopefully early next week but not 100% sure with my time issues.

 

[mdevilz]

I havent had a chance to setup a test environment yet. Hopefully early next week but not 100% sure with my time issues.

Hi Ksosez, I was wondering if you had a change to test this?

 

[ksosez]

Hi Ksosez, I was wondering if you had a change to test this?

I havent sorry, Hopefully this coming week I will. But no promises, if anyone else has any time you are welcome to play with it a bit.

 

[billthebull]

Hi,
I added the following line to my cluster.conf

<cman keyfile="/var/lib/pve-cluster/corosync.authkey" transport="udpu"/>

but it seems not to work. I got parsing error messages so it looks that there is a syntax error inside the cluster.conf....

Here is my complete cluster.conf:
root@proxmox40:~# cat /etc/pve/cluster.conf
<?xml version="1.0"?>
<cluster name="newcluster" config_version="1">

<cman keyfile="/var/lib/pve-cluster/corosync.authkey" transport="udpu"/>

</cman>

<clusternodes>
<clusternode name="proxmox40" votes="1" nodeid="1"/>
</clusternodes>

</cluster>

Here is the output:
root@proxmox40:~# /etc/init.d/cman start
Starting cluster:
Checking if cluster has been disabled at boot... [ OK ]
Checking Network Manager... [ OK ]
Global setup... [ OK ]
Loading kernel modules... [ OK ]
Mounting configfs... [ OK ]
Starting cman... /etc/cluster/cluster.conf:6: parser error : Opening and ending tag mismatch: cluster line 2 and cman
</cman>
^
/etc/cluster/cluster.conf:8: parser error : Extra content at the end of the document
<clusternodes>
^

Unable to get the configuration
/etc/cluster/cluster.conf:6: parser error : Opening and ending tag mismatch: cluster line 2 and cman
</cman>
^
/etc/cluster/cluster.conf:8: parser error : Extra content at the end of the document
<clusternodes>
^
corosync [MAIN ] Corosync Cluster Engine ('1.4.3'): started and ready to provide service.
corosync [MAIN ] Corosync built-in features: nss
corosync [MAIN ] Unable to read config from /etc/cluster/cluster.conf
corosync [MAIN ] Corosync Cluster Engine exiting with status 8 at main.c:1665.
corosync died: Could not read cluster configuration Check cluster logs for details
[FAILED]


Would be great if somebody can help me.

Thanke!!




​

 

[dietmar]

<?xml version="1.0"?>
<cluster name="newcluster" config_version="1">

<cman keyfile="/var/lib/pve-cluster/corosync.authkey" transport="udpu"/>
</cman>
​

You close the cman tag twice!

 

[billthebull]

You close the cman tag twice!

ok... means... I have to remove the "/", right?

<cman keyfile="/var/lib/pve-cluster/corosync.authkey" transport="udpu">

I also have problems editing this file. Only one time, directly after configuring a cluster I am allowed to
overwrite the cluster.config file. Later it is not possible. Is there a trick?

Thanks!

 

[dietmar]

I also have problems editing this file. Only one time, directly after configuring a cluster I am allowed to
overwrite the cluster.config file. Later it is not possible. Is there a trick?

You can't edit that file if you cluster is not quorate.
Does it help if you set the expected votes to 1:

# pvecm expected 1

reboot after you corrected the bug.