Problem adding node to Cluster

[Scott Alden]

I'm hanging on this error message when trying to add a brand new node to a cluster:

Starting cluster:
Checking if cluster has been disabled at boot... [ OK ]
Checking Network Manager... [ OK ]
Global setup... [ OK ]
Loading kernel modules... [ OK ]
Mounting configfs... [ OK ]
Starting cman... [ OK ]
Waiting for quorum... Timed-out waiting for cluster

I've checked my firewall settings, and I believe that every port is open that's needed for operation, and multicast is activated.

Anyone have any suggestions on how to get this working?

 

[Scott Alden]

Update: I cancelled the 'add' (via ctrl-c) then issued the command 'pvecm expected 1', and then readded the node - I got a success message, however it seems that the members of the cluster cannot see each other, and I cannot access the external nodes via the web GUI - it pops up a prompt asking for a username/password but gives an error message "Permission Denied: Invalid Ticket (401)"

root@frank:~# pvecm nodes
Node  Sts   Inc   Joined               Name
   1   M    608   2012-05-02 19:31:41  frank
   2   X      0                        euphrates

root@frank:~# pvecm s
Version: 6.2.0
Config Version: 2
Cluster Name: servercluster
Cluster Id: 56884
Cluster Member: Yes
Cluster Generation: 608
Membership state: Cluster-Member
Nodes: 1
Expected votes: 1
Total votes: 1
Node votes: 1
Quorum: 1  
Active subsystems: 5
Flags: 
Ports Bound: 0  
Node name: frank
Node ID: 1
Multicast addresses: 239.192.222.19 
Node addresses: xx.xx.xx.162

root@euphrates:~# pvecm nodes
Node  Sts   Inc   Joined               Name
   1   X      0                        frank
   2   M   2072   2012-05-02 20:35:54  euphrates

root@euphrates:~# pvecm s
Version: 6.2.0
Config Version: 2
Cluster Name: servercluster
Cluster Id: 56884
Cluster Member: Yes
Cluster Generation: 2300
Membership state: Cluster-Member
Nodes: 1
Expected votes: 1
Total votes: 1
Node votes: 1
Quorum: 1  
Active subsystems: 5
Flags: 
Ports Bound: 0  
Node name: euphrates
Node ID: 2
Multicast addresses: 239.192.222.19 
Node addresses: xx.xx.xx.141

So I'm stumped... any help would be appreciated.

 

[Scott Alden]

OK one more update - I rebooted the nodes, and they were in sync for about 5 minutes where it all seemed like it was working properly... This didn't last though, and now the syslog is flooded with this message:

May  2 23:02:26 frank pmxcfs[1598]: [status] crit: cpg_send_message failed: 9

Is Proxmox 2.1 ready for production? Is anyone using this in a live environment? I could go back to 1.9 which works fantastically for us.

 

[dietmar]

Sems cman is not running. Are there any other cman related hints in /var/log/syslog?

 

[Scott Alden]

Coming back to this after a few days off - I'm still not able to have the clusters see each other. I've opened ports 5404, and 5405 as well as the multicast address. Has anyone gotten this working with a firewall in place?