Quorum Activity blocked

D

djahida

Active Member
Mar 24, 2015
44
0
26
Hi,
I have trouble on my cluster pve 4.3 with 3 nodes.
Two nodes are seeing each other, but the third doesn’t see the two others nodes.
Some advice, please, how to resolve this problem.


1. /etc/pve/corosync is readonly on the node 2.
2. pvecm s on node 1 and 3
root@node3 : ~ # pvecm s
Quorum information
------------------
Date: Sat Jun 24 14:35:43 2017
Quorum provider: corosync_votequorum
Nodes: 2
Node ID: 0x00000003
Ring ID: 3/2004652
Quorate: Yes

Votequorum information
----------------------
Expected votes: 3
Highest expected: 3
Total votes: 2
Quorum: 2
Flags: Quorate

Membership information
----------------------
Nodeid Votes Name
0x00000003 1 node3 (local)
0x00000001 1 node1

3. pvecm s on node 2
root@node 2: ~ # pvecm s
Quorum information
------------------
Date: Sat Jun 24 15:16:03 2017
Quorum provider: corosync_votequorum
Nodes: 1
Node ID: 0x00000002
Ring ID: 2/1841220
Quorate: No

Votequorum information
----------------------
Expected votes: 3
Highest expected: 3
Total votes: 1
Quorum: 2 Activity blocked
Flags:

Membership information
----------------------
Nodeid Votes Name
0x00000002 1 node 2 (local)


4. Log on node 1
root@node1: /etc # tail -f /var/log/syslog
J
Jun 24 15:08:44 node1corosync[3107]: [MAIN ] Completed service synchronization, ready to provide service.
Jun 24 15:08:47 node1corosync[3107]: [TOTEM ] A new membership (node3:2008392) was formed. Members
Jun 24 15:08:47 node1corosync[3107]: [QUORUM] Members[2]: 3 1
Jun 24 15:08:47 node1corosync[3107]: [MAIN ] Completed service synchronization, ready to provide service.
Jun 24 15:08:49 node1corosync[3107]: [TOTEM ] A new membership (node3:2008396) was formed. Members
Jun 24 15:08:49 node1corosync[3107]: [QUORUM] Members[2]: 3 1

5. Log on node 2
Jun 24 14:35:11 node2 pve-ha-lrm[3225]: ipcc_send_rec failed: Transport endpoint is not connected
Jun 24 14:35:11 node2 pve-ha-crm[3213]: ipcc_send_rec failed: Transport endpoint is not connected
Jun 24 14:35:11 node2 PP007 pve-ha-crm[3213]: ipcc_send_rec failed: Connection refused
Jun 24 14:35:11 node2 pve-ha-lrm[3225]: ipcc_send_rec failed: Connection refused
Jun 24 14:35:11 node2 pve-ha-crm[3213]: ipcc_send_rec failed: Connection refused
Jun 24 14:35:11 node2 pve-ha-lrm[3225]: ipcc_send_rec failed: Connection refused
Jun 24 14:35:12 node2 pmxcfs[4909]: [main] notice: exit proxmox configuration filesystem (0)
Jun 24 14:35:12 node2 systemd[1]: Starting The Proxmox VE cluster filesystem...
Jun 24 14:35:12 node2 pveproxy[4352]: ipcc_send_rec failed: Transport endpoint is not connected
Jun 24 14:35:12 node2 pveproxy[4352]: ipcc_send_rec failed: Connection refused
Jun 24 14:35:12 node2 pveproxy[4352]: ipcc_send_rec failed: Connection refused

6. Log on node 3
root@node3 : ~ # tail -f /var/log/syslog
Jun 24 15:10:52 node3 corosync[18114]: [MAIN ] Completed service synchronization, ready to provide service.
Jun 24 15:10:54 node3 corosync[18114]: [TOTEM ] A new membership (node3:2008632) was formed. Members
Jun 24 15:10:54 node3 corosync[18114]: [QUORUM] Members[2]: 3 1
Jun 24 15:10:54 node3 corosync[18114]: [MAIN ] Completed service synchronization, ready to provide service.
Jun 24 15:10:56 node3 corosync[18114]: [TOTEM ] A new membership (node3 :2008636) was formed. Members
Jun 24 15:10:56 node3 corosync[18114]: [QUORUM] Members[2]: 3 1

Thank's
 
Seems pmxcfs was stopped by someone (on node2)? Or what happened on node2 by that time? I would reboot to resolve that cleanly.
 
Hello,
I have similar problem on my simple setup. I want to build a cluster and i cannot. I have only 1 network and the latest proxmox install.
=============================================================================
NODE1: (worked OK, i have set up a new cluster on it)
Cluster information
-------------------
Name: dellcluster
Config Version: 6
Transport: knet
Secure auth: on

Quorum information
------------------
Date: Fri Jul 8 18:48:37 2022
Quorum provider: corosync_votequorum
Nodes: 1
Node ID: 0x00000001
Ring ID: 1.a
Quorate: No

Votequorum information
----------------------
Expected votes: 2
Highest expected: 2
Total votes: 1
Quorum: 2 Activity blocked
Flags:

Membership information
----------------------
Nodeid Votes Name
0x00000001 1 192.168.1.40 (local)
-----------------------------------------------------------------------------------------------------
NODE2: (which cannot join the cluster and gets broken)
Cluster information
-------------------
Name: dellcluster
Config Version: 6
Transport: knet
Secure auth: on

Quorum information
------------------
Date: Fri Jul 8 18:48:14 2022
Quorum provider: corosync_votequorum
Nodes: 1
Node ID: 0x00000002
Ring ID: 2.a
Quorate: No

Votequorum information
----------------------
Expected votes: 2
Highest expected: 2
Total votes: 1
Quorum: 2 Activity blocked
Flags:

Membership information
----------------------
Nodeid Votes Name
0x00000002 1 192.168.1.30 (local)
=============================================
On node2 the syslog looks like:
Jul 8 19:16:38 proxmox2 pveproxy[1414]: starting 1 worker(s)
Jul 8 19:16:38 proxmox2 pveproxy[1414]: worker 12737 started
Jul 8 19:16:38 proxmox2 pveproxy[12736]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/API
Server/AnyEvent.pm line 1943.
Jul 8 19:16:38 proxmox2 pveproxy[12737]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/API
Server/AnyEvent.pm line 1943.
Jul 8 19:16:38 proxmox2 pveproxy[1414]: worker 12732 finished
Jul 8 19:16:38 proxmox2 pveproxy[1414]: starting 1 worker(s)
Jul 8 19:16:38 proxmox2 pveproxy[1414]: worker 12738 started
Jul 8 19:16:38 proxmox2 pveproxy[12738]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/API
Server/AnyEvent.pm line 1943.
-------------------------------------------------
what key? why? on new setup from scratch?
root@proxmox2:/etc/pve# ls -la
total 5
drwxr-xr-x 2 root www-data 0 Jan 1 1970 .
drwxr-xr-x 90 root root 4096 Jul 8 17:06 ..
-r--r----- 1 root www-data 591 Jan 1 1970 .clusterlog
-r--r----- 1 root www-data 455 Jul 8 18:02 corosync.conf
-rw-r----- 1 root www-data 2 Jan 1 1970 .debug
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 local -> nodes/proxmox2
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 lxc -> nodes/proxmox2/lxc
-r--r----- 1 root www-data 237 Jan 1 1970 .members
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 openvz -> nodes/proxmox2/openvz
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 qemu-server -> nodes/proxmox2/qemu-server
-r--r----- 1 root www-data 222 Jan 1 1970 .rrd
-r--r----- 1 root www-data 710 Jan 1 1970 .version
-r--r----- 1 root www-data 18 Jan 1 1970 .vmlist
------------
i see that the destinations for these 4 links do NOT exist!!!! There are NO subdirectories in /etc/pve on the second node which i wanted to add to the cluster.
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 local -> nodes/proxmox2
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 lxc -> nodes/proxmox2/lxc
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 openvz -> nodes/proxmox2/openvz
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 qemu-server -> nodes/proxmox2/qemu-server

if this is not normal, then this is the proxmox code which is broken on proxmox 7.2.1.
Could you fix it, please?
what to do next?
Thanks.
 
You must log in or register to reply here.
Top Bottom
Back