Understanding cluster.conf

ksosez

Guest
Hey there,

I am trying to understand cluster.conf and its relationship to corosync.

It appears from here I can do totem settings in the cluster.conf, can I do more advanced things to corosync such as using unicast as referenced here and in the /usr/share/doc/corosync-pve/examples/corosync.conf.example.udpu


I would really like to turn on the udpu/unicast option since Singlehop does not support unicast and I would like to manage all the systems from one interface.
 

e100

Renowned Member
Nov 6, 2010
1,248
33
68
Columbus, Ohio
ulbuilder.wordpress.com
I am the one who wrote this: http://pve.proxmox.com/wiki/Multicast_notes#Changing_netmtu

According to the docs I have read yes you can configure lots of stuff in the cluster config, not just totem.
I also made the note on the wiki that I have not tested that yet.

Code:
man 5 cman

The info here looks promising:
https://bugzilla.redhat.com/show_bug.cgi?id=657041

Notes from that page:
1. To enable the UDPU transport, manually add the line <cman transport="udpu"/> to the cluster.conf file:
Code:
<cman [B]transport="udpu"[/B] keyfile="/var/lib/pve-cluster/corosync.authkey">
2. You can not have a <totem> tag in your cluster.conf if you also want udpu.
3. You must restart the whole cluster after making the change

One last note, the issue with the totem tag and udpu issue was recently fixed, maybe Proxmox has the fix already?
https://bugzilla.redhat.com/show_bug.cgi?id=695795

I tested the change to my cluster.conf and it does seem to work:
/var/log/cluster/corosync.log:
Code:
Feb 01 22:19:05 corosync [TOTEM ] Initializing transport (UDP/IP [B]Unicast[/B]).
 
Last edited:

ksosez

Guest
Thanks a lot e100,

Maybe we should add this to the wiki?

I still need to figure out how to specify the other IP's of the other systems.
 
Last edited by a moderator:

dietmar

Proxmox Staff Member
Staff member
Apr 28, 2005
17,073
488
103
Austria
www.proxmox.com
My understanding is that udpu uses broadcast to communicate so it might not be necessary to specify the other addresses.

AFAIK udpu only use unicast. See 'man corosync.conf':
Code:
              To avoid the use of  multicast  entirely,  a  unicast  transport
              parameter "udpu" can be specified.  This requires specifying the
              list of members that could potentially make  up  the  membership
              before deployment.
 

ksosez

Guest
I havent had a chance to setup a test environment yet. Hopefully early next week but not 100% sure with my time issues.
 

ksosez

Guest
Hi Ksosez, I was wondering if you had a change to test this?

I havent sorry, Hopefully this coming week I will. But no promises, if anyone else has any time you are welcome to play with it a bit.
 

billthebull

Member
Oct 4, 2012
8
0
21
Hi,
I added the following line to my cluster.conf

<cman keyfile="/var/lib/pve-cluster/corosync.authkey" transport="udpu"/>

but it seems not to work. I got parsing error messages so it looks that there is a syntax error inside the cluster.conf....

Here is my complete cluster.conf:
root@proxmox40:~# cat /etc/pve/cluster.conf
<?xml version="1.0"?>
<cluster name="newcluster" config_version="1">

<cman keyfile="/var/lib/pve-cluster/corosync.authkey" transport="udpu"/>

</cman>

<clusternodes>
<clusternode name="proxmox40" votes="1" nodeid="1"/>
</clusternodes>

</cluster>

Here is the output:
root@proxmox40:~# /etc/init.d/cman start
Starting cluster:
Checking if cluster has been disabled at boot... [ OK ]
Checking Network Manager... [ OK ]
Global setup... [ OK ]
Loading kernel modules... [ OK ]
Mounting configfs... [ OK ]
Starting cman... /etc/cluster/cluster.conf:6: parser error : Opening and ending tag mismatch: cluster line 2 and cman
</cman>
^
/etc/cluster/cluster.conf:8: parser error : Extra content at the end of the document
<clusternodes>
^

Unable to get the configuration
/etc/cluster/cluster.conf:6: parser error : Opening and ending tag mismatch: cluster line 2 and cman
</cman>
^
/etc/cluster/cluster.conf:8: parser error : Extra content at the end of the document
<clusternodes>
^
corosync [MAIN ] Corosync Cluster Engine ('1.4.3'): started and ready to provide service.
corosync [MAIN ] Corosync built-in features: nss
corosync [MAIN ] Unable to read config from /etc/cluster/cluster.conf
corosync [MAIN ] Corosync Cluster Engine exiting with status 8 at main.c:1665.
corosync died: Could not read cluster configuration Check cluster logs for details
[FAILED]


Would be great if somebody can help me.

Thanke!!




 

billthebull

Member
Oct 4, 2012
8
0
21
You close the cman tag twice!

ok... means... I have to remove the "/", right?

<cman keyfile="/var/lib/pve-cluster/corosync.authkey" transport="udpu">

I also have problems editing this file. Only one time, directly after configuring a cluster I am allowed to
overwrite the cluster.config file. Later it is not possible. Is there a trick?

Thanks!
 

dietmar

Proxmox Staff Member
Staff member
Apr 28, 2005
17,073
488
103
Austria
www.proxmox.com

I also have problems editing this file. Only one time, directly after configuring a cluster I am allowed to
overwrite the cluster.config file. Later it is not possible. Is there a trick?

You can't edit that file if you cluster is not quorate.
Does it help if you set the expected votes to 1:

# pvecm expected 1

reboot after you corrected the bug.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!