Messed up cluster setup

[idic]

So, i've been working on setting up a cluster with 2 home servers and a raspberry pi as a qdevice. I've messed it up a few times and fixed it each time with a ton of hassle. Right now it's working but I don't think i really fixed it properly last time and it's showing conflicting stuff now.

The last time I did a change was to change my corosync.conf file so I could change the network my cluster traffic would go over. I followed the online directions to add a second ring interface entry but it rejected that messed everything up. I was trying to change it from my prod 192.168.1.x network to a isolated 10.10.60.x network. I eventually got it working back on the initial network.

Now in my gui it is showing the cluster is on the 10.10.60.x interfaces.

Where my pvecm status shows

    [root@prox|11:35:54 ~]# pvecm status
    Cluster information
    -------------------
    Name:             ggCluster
    Config Version:   12
    Transport:        udpu
    Secure auth:      on
    
    Quorum information
    ------------------
    Date:             Mon Apr  6 11:36:51 2020
    Quorum provider:  corosync_votequorum
    Nodes:            2
    Node ID:          0x00000001
    Ring ID:          1.1b8
    Quorate:          Yes
    
    Votequorum information
    ----------------------
    Expected votes:   3
    Highest expected: 3
    Total votes:      2
    Quorum:           2
    Flags:            Quorate
    
    Membership information
    ----------------------
        Nodeid      Votes Name
    0x00000001          1 192.168.1.2 (local)
    0x00000002          1 192.168.1.21

And my messed up configuration files show

    [root@prox|13:55:02 /etc/pve]# cat corosync.conf
    logging {
      debug: off
      to_syslog: yes
    }
   
    nodelist {
      node {
        name: ggprox02
        nodeid: 2
        quorum_votes: 1
        ring0_addr: 10.10.60.10
      }
      node {
        name: prox
        nodeid: 1
        quorum_votes: 1
        ring0_addr: 10.10.60.11
      }
    }
   
    quorum {
      provider: corosync_votequorum
    }
   
    totem {
      cluster_name: ggCluster
      config_version: 12
      interface {
        linknumber: 0
      }
      ip_version: ipv4-6
      secauth: on
      transport: udpu
      version: 2
    }

and

    [root@prox|13:55:06 /etc/pve]# cat /etc/corosync/corosync.conf
    logging {
      debug: off
      to_syslog: yes
    }
   
    nodelist {
      node {
        name: ggprox02
        nodeid: 2
        quorum_votes: 1
        ring0_addr: 192.168.1.21
      }
      node {
        name: prox
        nodeid: 1
        quorum_votes: 1
        ring0_addr: 192.168.1.2
      }
    }
   
    quorum {
      device {
        model: net
        net {
          algorithm: ffsplit
          host: 192.168.1.26
          tls: on
        }
        votes: 1
      }
      provider: corosync_votequorum
    }

To just confirm, i snooped the traffic and it is running on the 192.168.1.x network.

I'm very worried about doing anything at this point as when I've followed the instructions (as i've understood them) all hell has broken loose. I need to add the raspberry pi in to make the quorum have 3 entries but I want it to be on the separate network.

Is there a way I can fix this without doing a reinstall?

thanks

 

[wolfgang]

Hi,

the problem I guess is you edit the /etc/corosync/corosync.conf on each node.
You should not edit it here. Always edit it in /etc/pve/corosync.conf.
It is working because you did not restart the corosync service on the node.
Edit the config in the /etc/pve and the changes will automatically be distributed to the cluster members.
Also, the config will be applied on all members.