Fixing Corosync, Can I do this?

[Nefariousparity]

Here is my corosync config file, ring0 I have changed so my public internet ips are not so obvious. Can I swap ring0 and ring1? So instead of primary being public network it is a private network. I'm hoping to stop these errors. All nodes are, "Virtual Environment 7.3-3"

If so is this the correct processes?

Edit /etc/corosync/corosync.conf

To look something like this,

nodelist {
node {
name: pve-j-dal
nodeid: 1
quorum_votes: 1
ring0_addr: 10.10.11.6
ring1_addr: 8.8.8.8
}

systemctl coronsync restart?

Of course edit all the configurations, and restart on each node.

Dec 27 02:02:38 pve-j-dal corosync[1688754]: [KNET ] link: host: 2 link: 1 is down
Dec 27 02:02:38 pve-j-dal corosync[1688754]: [KNET ] host: host: 2 (passive) best link: 0 (pri: 1)
Dec 27 02:02:41 pve-j-dal corosync[1688754]: [KNET ] rx: host: 2 link: 1 is up
Dec 27 02:02:41 pve-j-dal corosync[1688754]: [KNET ] link: Resetting MTU for link 1 because host 2 joined
Dec 27 02:02:41 pve-j-dal corosync[1688754]: [KNET ] host: host: 2 (passive) best link: 0 (pri: 1)
Dec 27 02:02:41 pve-j-dal corosync[1688754]: [KNET ] pmtud: Global data MTU changed to: 1397
Dec 27 02:03:51 pve-j-dal corosync[1688754]: [KNET ] link: host: 4 link: 1 is down
Dec 27 02:03:51 pve-j-dal corosync[1688754]: [KNET ] link: host: 3 link: 1 is down
Dec 27 02:03:51 pve-j-dal corosync[1688754]: [KNET ] link: host: 2 link: 1 is down
Dec 27 02:03:51 pve-j-dal corosync[1688754]: [KNET ] host: host: 4 (passive) best link: 0 (pri: 1)
Dec 27 02:03:51 pve-j-dal corosync[1688754]: [KNET ] host: host: 3 (passive) best link: 0 (pri: 1)
Dec 27 02:03:51 pve-j-dal corosync[1688754]: [KNET ] host: host: 2 (passive) best link: 0 (pri: 1)
Dec 27 02:03:54 pve-j-dal corosync[1688754]: [KNET ] rx: host: 4 link: 1 is up
Dec 27 02:03:54 pve-j-dal corosync[1688754]: [KNET ] link: Resetting MTU for link 1 because host 4 joined
Dec 27 02:03:54 pve-j-dal corosync[1688754]: [KNET ] host: host: 4 (passive) best link: 0 (pri: 1)
Dec 27 02:03:54 pve-j-dal corosync[1688754]: [KNET ] rx: host: 3 link: 1 is up
Dec 27 02:03:54 pve-j-dal corosync[1688754]: [KNET ] link: Resetting MTU for link 1 because host 3 joined
Dec 27 02:03:54 pve-j-dal corosync[1688754]: [KNET ] host: host: 3 (passive) best link: 0 (pri: 1)
Dec 27 02:03:54 pve-j-dal corosync[1688754]: [KNET ] rx: host: 2 link: 1 is up
Dec 27 02:03:54 pve-j-dal corosync[1688754]: [KNET ] link: Resetting MTU for link 1 because host 2 joined
Dec 27 02:03:54 pve-j-dal corosync[1688754]: [KNET ] host: host: 2 (passive) best link: 0 (pri: 1)
Dec 27 02:03:54 pve-j-dal corosync[1688754]: [KNET ] pmtud: Global data MTU changed to: 1397

logging {
debug: off
to_syslog: yes
}

nodelist {
node {
name: pve-j-dal
nodeid: 1
quorum_votes: 1
ring0_addr: 8.8.8.8
ring1_addr: 10.10.11.6
}
node {
name: pve-k-dal
nodeid: 2
quorum_votes: 1
ring0_addr: 8.8.8.7
ring1_addr: 10.10.11.230
}
node {
name: pve-l-dal
nodeid: 3
quorum_votes: 1
ring0_addr: 8.8.8.6
ring1_addr: 10.10.11.231
}
node {
name: pve-m-dal
nodeid: 4
quorum_votes: 1
ring0_addr: 8.8.8.5
ring1_addr: 10.10.11.232
}
}

quorum {
provider: corosync_votequorum
}

totem {
cluster_name: bart2-dal
config_version: 4
interface {
linknumber: 0
}
interface {
linknumber: 1
}
ip_version: ipv4-6
link_mode: passive
secauth: on
version: 2
}

Thanks for looking!

 

[mira]

Please provide the output of pveversion -v.
You can specify the priority of each link with `knet_link_priority`. Add it below the `linknumber` line.
Links with a higher number have higher priority.

See also `man corosync.conf`

 

[Nefariousparity]

Hi Mira,

root@pve-j-dal:~# pveversion -v
proxmox-ve: 7.3-1 (running kernel: 5.15.35-1-pve)
pve-manager: 7.3-3 (running version: 7.3-3/c3928077)
pve-kernel-5.15: 7.2-14
pve-kernel-helper: 7.2-14
pve-kernel-5.13: 7.1-9
pve-kernel-5.11: 7.0-10
pve-kernel-5.4: 6.4-5
pve-kernel-5.15.74-1-pve: 5.15.74-1
pve-kernel-5.15.39-1-pve: 5.15.39-1
pve-kernel-5.15.35-2-pve: 5.15.35-5
pve-kernel-5.15.35-1-pve: 5.15.35-3
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-3-pve: 5.13.19-7
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.13.19-1-pve: 5.13.19-3
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.11.22-3-pve: 5.11.22-7
pve-kernel-5.4.128-1-pve: 5.4.128-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-5.4.78-2-pve: 5.4.78-2
pve-kernel-4.15: 5.4-19
pve-kernel-4.15.18-30-pve: 4.15.18-58
pve-kernel-4.15.18-12-pve: 4.15.18-36
ceph-fuse: 14.2.21-1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.2-5
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.2-8
libpve-guest-common-perl: 4.2-3
libpve-http-server-perl: 4.1-5
libpve-storage-perl: 7.2-12
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.0-3
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.2.7-1
proxmox-backup-file-restore: 2.2.7-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.0-1
proxmox-widget-toolkit: 3.5.3
pve-cluster: 7.3-1
pve-container: 4.4-2
pve-docs: 7.3-1
pve-edk2-firmware: 3.20220526-1
pve-firewall: 4.2-7
pve-firmware: 3.5-6
pve-ha-manager: 3.5.1
pve-i18n: 2.8-1
pve-qemu-kvm: 7.1.0-4
pve-xtermjs: 4.16.0-1
qemu-server: 7.3-1
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+2
vncterm: 1.7-1
zfsutils-linux: 2.1.6-pve1

 

[Nefariousparity]

Mira, so when I add "knet_link_priority" is this allowing me to choose the priority of the
ring0_addr: 8.8.8.8
ring1_addr: 10.10.11.6

So I can tell these nodes to sync via private network vrs public?

Or is this for setting the priority of the node itself? I am trying to tell it to do this over the private network as priority first.

 

[mira]

This is for the link, you add them to each of the interfaces:

interface {
linknumber: 0
knet_link_priority: 0
}
interface {
linknumber: 1
knet_link_priority: 10
}

This means that link 1 will be the preferred one if possible on all nodes in the cluster.

And don't forget to increment the config version! [0]


[0] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_corosync_configuration

 

[Nefariousparity]

Mira, I am still confused, can you show me a example with that in my above posted configuration please?

 

[vesalius]

Look at the bottom of you /etc/corosync/corosync.conf file. Even the one posted above find the totem section, then under that the interface subsection for for your link0 and link1 lines.

add the knet_link_priority: lined as @mira stated and Corosync should now use your private ring1 as primary.

 

[Nefariousparity]

@vesalius Thank you for the response. After I edit the corosync configuration on each node, I need to restart Coro sync? And this need to be done on each node one at a time?

 

[vesalius]

@vesalius Thank you for the response. After I edit the corosync configuration on each node, I need to restart Coro sync? And this need to be done on each node one at a time?

Read the last line of @mira post above and follow the directions from that link on how to edit. This link will also tell how to work around that fact that edits take effect immediately. As he stated don't forget to increment the config_version (currently at 4 above, increase to 5) when you save the other edits.

 

[Nefariousparity]

@vesalius Thank you and @mira . I think I have reached success based on your notes here, and reading the document on how to implement the changes. Am I correct that this document will push these changes to all nodes?

**Edit, I can actually see the incremented version appear in the logs.

 

[vesalius]

Correct and Congrats