Corosync Redundant Ring is not working

[ogusis01]

Hello I'm Oguz.

I have some issue about Corosync...
First of all i'm going to explain my network system.

1) I created 4 virtual box ubuntu computer and i configured them network addresses & routes with netplan yaml for ring topology.
Ring 1:
PC1: 192.168.10.101
PC2: 192.168.10.102
PC3: 192.168.10.103
PC4: 192.168.10.104
Ring 2:
PC1: 192.168.11.101
PC2: 192.168.11.102
PC3: 192.168.11.103
PC4: 192.168.11.104
Route example for PC1 Ring1:
-to 192.168.10.102
via 192.168.10.101
-to 192.168.10.103
via 192.168.10.102
-to 192.168.10.104
via 192.168.10.102

I configured route addresses for all computers and i can ping all nodes with any node. Also i tested system about working as ring. For example if a ping to PC4 from PC1, i can see ping activity in PC2 and PC3 with "tcpdump".

2) My goal is that the second ring will be automatically activated by corosync when any node falls from the first ring.

3) Corosync configuration of all computers:
totem {
version: 2
ip_version: ipv4-6
config_version: 10
cluster_name: mycluster
link_mode: passive
interface {
linknumber:0
knet_link_priority: 255
}
interface {
linknumber:1
knet_link_priority: 120
}
nodelist {
node {
ring0_addr: 192.168.10.101
ring1_addr: 192.168.11.101
quorum_votes: 1
name: master
nodeid:1
}
node {
ring0_addr: 192.168.10.102
ring1_addr: 192.168.11.102
quorum_votes: 1
name: node2
nodeid:2
}
node {
ring0_addr: 192.168.10.103
ring1_addr: 192.168.11.103
quorum_votes: 1
name: node3
nodeid:3
}
node {
ring0_addr: 192.168.10.103
ring1_addr: 192.168.11.103
quorum_votes: 1
name: node4
nodeid:4
}
quorum {
provider: corosync_votequorum
}

logging {
to_logfile: yes
logfile: /var/log/corosync/corosync.log
to_syslog: yes
timestamp: on
debug: off
}


4) Verification of corosync

4.1) I can see all nodes online of rings with "corosync-cfgtool -s" and "corosync-cmapctl | grep members".



4.2) Pacemaker shows all nodes online


5) After the verification, When pinging PC4 from PC1, I drop the NIC that belongs to PC3 and is also connected to ring 1 with "ifconfig enp0s9 down".

6) In the Corosync log file, it is written during the operations that link 0 is down and the best link that can be used in the current situation is link 1.



7) But ring 1 is never automatically activating by corosync.

I can not describe problem of my system. Please help me...

 

[fabian]

what do you mean with activating ring 1? it does show link 0 going down, and that the remaining link 1 is the best choice now..

 

[ogusis01]

what do you mean with activating ring 1? it does show link 0 going down, and that the remaining link 1 is the best choice now..

Yes, it shows that it is, but when ring 0 drops, ring 1 does not automatically activate. I wonder to cause of this.

What I mean by active. When Ring 0 is disconnected, the work done by Ring 0 before the connection is disconnected is transferred to Ring 1. Just like in classical redundant ring protocols.

 

[fabian]

corosync only handles its own traffic, it does not magically re-route other traffic if that's what you expected...

 

[ogusis01]

corosync only handles its own traffic, it does not magically re-route other traffic if that's what you expected.

Do you know any program or way that I can provide a redundant network structure in ubuntu for the entire traffic?

 

[fabian]

usually that would be handled via bonding of NICs

 

[ogusis01]

usually that would be handled via bonding of NICs

I accomplished my goal with NIC bonding. Thanks for help.
So, do you think corosync can be configured to provide ethernet redundancy without making any changes to user applications in the operating system? Have you ever come across anyone doing this before?

 

[fabian]

corosync does have a feature where you can define "virtual" NICs (they call that "nozzle" after the library they use for it) that use the corosync networks for routing traffic - but I have no experience with that, nor do we use that in PVE.