So I ran into a weird situation today. Something went haywire on the switch stack we are using for bonding taking out one node in a four node HA cluster. The node shows links as up but traffic goes nowhere. The three other nodes are fine. We are using a separate network for corosync so as far as corosync was concerned the node wasn't down however the Ceph and the network storage we use are on different networks and so were unreachable. The node never triggered as down even though it was unreachable. I am wondering if I have everything configured correctly at this point. Here is the config (yes, I know the 169.254 network needs changing, that was on my list to do today when this happened, wasn't my decision initially)
/etc/network/interfaces
/etc/pve/corosync.conf
vmbr0 is the public facing IP which also mounts the NFS storage we are using for backup,snippets,iso,vztmpl
bond1 is dedicated to Ceph and used as a second ring for corosync (Since the corosync network is bonded this may be redundant but this cluster will host critical services and I want to have as much redundancy built in as possible)
bond2 dedicated to Corosync exclusively
These are the logs for what I think are relevant services after a reboot (journalctl -u corosync -u pmxcfs -u pve-ha-lrm): https://pastebin.com/4NCZeJvK
I was going to include pvestatd as well but that looked to be all complaining about the nfs storage not being reachable and didn't seem relevant. Please advise if I have the network configured incorrectly here or if I have corosync configured incorrectly. This is my first HA setup with Proxmox and I am still learning.
/etc/network/interfaces
/etc/pve/corosync.conf
vmbr0 is the public facing IP which also mounts the NFS storage we are using for backup,snippets,iso,vztmpl
bond1 is dedicated to Ceph and used as a second ring for corosync (Since the corosync network is bonded this may be redundant but this cluster will host critical services and I want to have as much redundancy built in as possible)
bond2 dedicated to Corosync exclusively
These are the logs for what I think are relevant services after a reboot (journalctl -u corosync -u pmxcfs -u pve-ha-lrm): https://pastebin.com/4NCZeJvK
I was going to include pvestatd as well but that looked to be all complaining about the nfs storage not being reachable and didn't seem relevant. Please advise if I have the network configured incorrectly here or if I have corosync configured incorrectly. This is my first HA setup with Proxmox and I am still learning.
