Host not connected to cluster?

qwaszx

New Member
Jan 29, 2009
3
0
1
Hi, I have an issue with one of the machines in my cluster: when I do pveca -l from the cluster master it says "14 : 10.0.0.51 N ERROR: 500 Can't connect to localhost:50011 (connect: Connection refused)". When I use any other node (10.0.0.2) it shows a successful connection. When I pveca -l from the node in question (10.0.0.51) it shows no errors. The web ui says "unable to load cluster table" beside the node. Is there a way I can fix this without taking the cluster down?

From the cluster master:
Code:
h1:~# pveca -l                                                   
CID----IPADDRESS----ROLE-STATE--------UPTIME---LOAD----MEM---ROOT---DATA
 1 : 10.0.0.1        M     S  103 days 21:38   0.63    19%     4%     1%
 2 : 10.0.0.12       N     A  103 days 21:49   0.00    22%     3%     1%
 3 : 10.0.0.11       N     A  103 days 21:49   0.01    23%     3%     1%
 4 : 10.0.0.10       N     A  103 days 21:48   0.00    23%     3%     1%
 5 : 10.0.0.9        N     A  103 days 21:48   0.00    22%     3%     1%
 6 : 10.0.0.8        N     A  103 days 21:48   0.00    22%     3%     1%
 7 : 10.0.0.7        N     A  103 days 21:48   0.05    21%     3%     1%
 8 : 10.0.0.6        N     A  103 days 21:48   0.00    21%     3%     1%
 9 : 10.0.0.5        N     A  103 days 21:48   0.01    21%     3%     1%
10 : 10.0.0.4        N     A  103 days 21:48   0.00    22%     3%     1%
11 : 10.0.0.3        N     A  103 days 21:42   0.01    22%     3%     2%
12 : 10.0.0.2        N     A  103 days 21:42   0.00    27%     3%     1%
14 : 10.0.0.51       N     ERROR: 500 Can't connect to localhost:50011 (connect: Connection refused)

15 : 10.0.0.31       N     A   97 days 21:12   0.04    13%     1%     0%
16 : 10.0.0.40       N     A   97 days 20:39   0.01    13%     4%     2%
17 : 10.0.0.39       N     A   97 days 20:08   0.00    13%     4%     2%
18 : 10.0.0.38       N     A   97 days 21:06   0.01    13%     4%     2%
19 : 10.0.0.37       N     A   97 days 21:26   0.02    12%     4%     2%
20 : 10.0.0.34       N     A   97 days 20:25   0.02    13%     4%     2%
21 : 10.0.0.36       N     A   97 days 21:24   0.00    12%     4%     2%
22 : 10.0.0.33       N     A   97 days 20:18   0.04    13%     4%     2%
23 : 10.0.0.32       N     A   97 days 19:57   0.01    13%     4%     2%
From a cluster member:
Code:
h2:~# pveca -l
CID----IPADDRESS----ROLE-STATE--------UPTIME---LOAD----MEM---ROOT---DATA
 1 : 10.0.0.1        M     S  103 days 21:40   0.41    17%     4%     1%
 2 : 10.0.0.12       N     A  103 days 21:50   0.00    22%     3%     1%
 3 : 10.0.0.11       N     A  103 days 21:50   0.00    24%     3%     1%
 4 : 10.0.0.10       N     A  103 days 21:50   0.00    23%     3%     1%
 5 : 10.0.0.9        N     A  103 days 21:50   0.00    22%     3%     1%
 6 : 10.0.0.8        N     A  103 days 21:50   0.00    22%     3%     1%
 7 : 10.0.0.7        N     A  103 days 21:50   0.04    22%     3%     1%
 8 : 10.0.0.6        N     A  103 days 21:50   0.00    21%     3%     1%
 9 : 10.0.0.5        N     A  103 days 21:50   0.00    21%     3%     1%
10 : 10.0.0.4        N     A  103 days 21:50   0.00    22%     3%     1%
11 : 10.0.0.3        N     A  103 days 21:43   0.00    22%     3%     2%
12 : 10.0.0.2        N     A  103 days 21:43   0.00    28%     3%     1%
14 : 10.0.0.51       N     A   46 days 22:48   0.00     3%     1%     0%
15 : 10.0.0.31       N     A   97 days 21:14   0.01    13%     1%     0%
16 : 10.0.0.40       N     A   97 days 20:40   0.00    13%     4%     2%
17 : 10.0.0.39       N     A   97 days 20:10   0.00    13%     4%     2%
18 : 10.0.0.38       N     A   97 days 21:08   0.00    13%     4%     2%
19 : 10.0.0.37       N     A   97 days 21:28   0.10    13%     4%     2%
20 : 10.0.0.34       N     A   97 days 20:26   0.00    13%     4%     2%
21 : 10.0.0.36       N     A   97 days 21:25   0.00    13%     4%     2%
22 : 10.0.0.33       N     A   97 days 20:19   0.01    13%     4%     2%
23 : 10.0.0.32       N     A   97 days 19:59   0.00    13%     4%     2%

From the node which the Master cannot connect to:
Code:
h51:~# pveca -l
CID----IPADDRESS----ROLE-STATE--------UPTIME---LOAD----MEM---ROOT---DATA
 1 : 10.0.0.1        M     S  103 days 21:45   0.42    17%     4%     1%
 2 : 10.0.0.12       N     A  103 days 21:55   0.01    22%     3%     1%
 3 : 10.0.0.11       N     A  103 days 21:55   0.00    23%     3%     1%
 4 : 10.0.0.10       N     A  103 days 21:55   0.00    22%     3%     1%
 5 : 10.0.0.9        N     A  103 days 21:55   0.00    22%     3%     1%
 6 : 10.0.0.8        N     A  103 days 21:55   0.00    22%     3%     1%
 7 : 10.0.0.7        N     A  103 days 21:55   0.01    22%     3%     1%
 8 : 10.0.0.6        N     A  103 days 21:55   0.00    22%     3%     1%
 9 : 10.0.0.5        N     A  103 days 21:55   0.00    22%     3%     1%
10 : 10.0.0.4        N     A  103 days 21:55   0.00    22%     3%     1%
11 : 10.0.0.3        N     A  103 days 21:49   0.00    22%     3%     2%
12 : 10.0.0.2        N     A  103 days 21:49   0.00    28%     3%     1%
14 : 10.0.0.51       N     A   46 days 22:53   0.00     4%     1%     0%
15 : 10.0.0.31       N     A   97 days 21:19   0.00    13%     1%     0%
16 : 10.0.0.40       N     A   97 days 20:45   0.00    13%     4%     2%
17 : 10.0.0.39       N     A   97 days 20:15   0.00    13%     4%     2%
18 : 10.0.0.38       N     A   97 days 21:13   0.06    13%     4%     2%
19 : 10.0.0.37       N     A   97 days 21:33   0.08    13%     4%     2%
20 : 10.0.0.34       N     A   97 days 20:31   0.00    13%     4%     2%
21 : 10.0.0.36       N     A   97 days 21:30   0.01    13%     4%     2%
22 : 10.0.0.33       N     A   97 days 20:24   0.00    13%     4%     2%
23 : 10.0.0.32       N     A   97 days 20:04   0.12    13%     4%     2%
 
check if /etc/pve/cluster.cfg is the same on the master and the out-of-sync node.

then please stop on both the following services.

  • /etc/init.d/pvetunnel stop
  • /etc/init.d/pvemirror stop
  • /etc/init.d/pvedaemon stop
and start again:

  • /etc/init.d/pvetunnel start
  • /etc/init.d/pvemirror start
  • /etc/init.d/pvedaemon start
 
The config files were identical but restarting on the cluster master fixed it. Thanks :-)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!