stop ring0 does not start redundant ring ring1

[Mobin]

We have a new setup of Proxmox cluster. Recently we configured redundant ring for corosync.

But when we are making down the first ring (ring0) that particular node is going out from the cluster, ie, cluster is not working through the redundant link.

pveversion
pve-manager/6.1-3/37248ce6 (running kernel: 5.3.10-1-pve)

pvecm status
Cluster information
-------------------
Name: PubWebApp-Clus2
Config Version: 8
Transport: knet
Secure auth: on

Quorum information
------------------
Date: Tue Feb 2 08:54:28 2021
Quorum provider: corosync_votequorum
Nodes: 6
Node ID: 0x00000003
Ring ID: 2.2598
Quorate: Yes

Votequorum information
----------------------
Expected votes: 6
Highest expected: 6
Total votes: 6
Quorum: 4
Flags: Quorate

Membership information
----------------------
Nodeid Votes Name
0x00000002 1 172.x.x.44
0x00000003 1 172.x.x.45 (local)
0x00000004 1 172.x.x.63
0x00000005 1 172.x.x.64
0x00000006 1 172.x.x.65
0x00000007 1 172.x.x.67

​

root@E1B15:~# cat /etc/pve/corosync.conf
logging {
debug: off
to_syslog: yes
}

nodelist {
node {
name: E1B14
nodeid: 2
quorum_votes: 1
ring0_addr: 172.x.x.44
ring1_addr: 172.x.x.44
}
node {
name: E1B15
nodeid: 3
quorum_votes: 1
ring0_addr: 172.x.x.45
ring1_addr: 172.x.x.45
}
node {
name: E2B13
nodeid: 4
quorum_votes: 1
ring0_addr: 172.x.x.63
ring1_addr: 172.x.x.63
}
node {
name: E2B14
nodeid: 5
quorum_votes: 1
ring0_addr: 172.x.x.64
ring1_addr: 172.x.x.64
}
node {
name: E2B15
nodeid: 6
quorum_votes: 1
ring0_addr: 172.18.x.65
ring1_addr: 172.18.x.65
}
node {
name: E4B15
nodeid: 7
quorum_votes: 1
ring0_addr: 172.18.x.67
ring1_addr: 172.18.x.67
}
}

quorum {
provider: corosync_votequorum
}

totem {
cluster_name: PubWebApp-Clus2
config_version: 8
interface {
linknumber: 0
}
interface {
linknumber: 1
}
ip_version: ipv4-6
secauth: on
version: 2
}

root@E1B15:~# cat /etc/corosync/corosync.conf
logging {
debug: off
to_syslog: yes
}

nodelist {
node {
name: E1B14
nodeid: 2
quorum_votes: 1
ring0_addr: 172.18.x.44
ring1_addr: 172.18.x.44
}
node {
name: E1B15
nodeid: 3
quorum_votes: 1
ring0_addr: 172.18.x.45
ring1_addr: 172.18.x.45
}
node {
name: E2B13
nodeid: 4
quorum_votes: 1
ring0_addr: 172.18.x.63
ring1_addr: 172.18.x.63
}
node {
name: E2B14
nodeid: 5
quorum_votes: 1
ring0_addr: 172.18.x.64
ring1_addr: 172.18.x.64
}
node {
name: E2B15
nodeid: 6
quorum_votes: 1
ring0_addr: 172.18.x.65
ring1_addr: 172.18.x.65
}
node {
name: E4B15
nodeid: 7
quorum_votes: 1
ring0_addr: 172.18.x.67
ring1_addr: 172.18.x.67
}
}

quorum {
provider: corosync_votequorum
}

totem {
cluster_name: PubWebApp-Clus2
config_version: 8
interface {
linknumber: 0
}
interface {
linknumber: 1
}
ip_version: ipv4-6
secauth: on
version: 2
}

​

 

[mira]

Try updating to the latest PVE version, since 6.1 there have been some fixes for libknet and pve-cluster.

 

[Mobin]

Could you please share the issue which is related to 6.1 and please suggest a stable proxmox version

 

[guletz]

ring0_addr: 172.18.x.44
ring1_addr: 172.18.x.44

Hi,

It do not seems to be a good ideea to have the same IP space(like 172.18.x.44) for the both rings(0 and 1)

From man corosync:

"...corosync resolves ringX_addr names/IP addresses using the getaddrinfo(3) call with respect of totem.ip_version setting.
getaddrinfo() function uses a sophisticated algorithm to sort node addresses into a preferred order and corosync always chooses
the first address in that list of the required family. As such it is essential that your DNS or /etc/hosts files are correctly configured
so that all addresses for ringX appear on the same network"


Good luck / Bafta!

 

[Mobin]

Hi,

Sorry it was typo mistake.

Both the ring has different IP Space

ring0_addr: 172.18.x.44
ring1_addr: 172.18.y.44

 

[mira]

PVE is kind of a rolling release, new packages containing new features, but also bug fixes are released constantly. The minor releases (PVE 6.3 has been released in November) are just 'snapshots' of those, and it is recommended to always keep up to date.

You can see a the changelog of packages with the following command: apt changelog <pkg-name> and also via the GUI (Node -> Updates -> Changelog).

Is there anything in the syslog indicating an issue with corosync?