Proxmox VE 5.1: Problem to change IP address class (C to B) on cluster node...

[Dubard]

Hi everybody,

I have 3 nodes (node1,node2 and node3) that make up my Proxmox cluster. I want to change my LAN IP adress range (class C to class B).

- I use a specific network interface for the 3 nodes that form my cluster for ZFS storage (network 10.12.1.0). The IP range does not change for this interface.
- The second network interface for each node is connected to my old LAN (192.168.1.0) and this is the one I want to change for all nodes.
I removed the node3 that i want reconfigured from the old LAN (192.168.1.0/24) to put it on the new LAN (172.12.12.0/16).
All network communications work because from the old LAN, I can join the new LAN without problems on all protocols.

I started by changing the IP address of a single node (node3) by just editing "/etc/network/interfaces" and "/etc/hosts" but, after interface restart, my node was no longer visible in my cluster !
I then decided to completely remove the node from my cluster by following https://pve.proxmox.com/wiki/Proxmox_VE_4.x_Cluster#Remove_a_cluster_node
...but I now have difficulties to reintegrate my node in my cluster.

When I execute the command below, the process is blocked at "waiting for quorum..." !

root@node3:~# systemctl stop pvestatd.service
root@node3:~# systemctl stop pvedaemon.service
root@node3:~# systemctl stop pve-cluster.service
root@node3:~#
root@node3:~# rm -rf /var/lib/pve-cluster
root@node3:~# rm -rf /etc/corosync
root@node3:~#
root@node3:~# systemctl start pve-cluster.service
root@node3:~#
root@node3:~# systemctl start pvestatd.service
root@node3:~# systemctl start pvedaemon.service
root@node3:~#
root@node3:~# pvecm add 10.12.1.3 -f
copy corosync auth key
stopping pve-cluster service
backup old database
Job for corosync.service failed because the control process exited with error code.
See "systemctl status corosync.service" and "journalctl -xe" for details.
waiting for quorum...

Also, i've this error with corosync service:

root@node3# systemctl status corosync.service
● corosync.service - Corosync Cluster Engine
   Loaded: loaded (/lib/systemd/system/corosync.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2018-03-20 10:55:12 CET; 4min 32s ago
     Docs: man:corosync
           man:corosync.conf
           man:corosync_overview
  Process: 50794 ExecStart=/usr/sbin/corosync -f $COROSYNC_OPTIONS (code=exited, status=20)
 Main PID: 50794 (code=exited, status=20)
      CPU: 119ms

Mar 20 10:55:12 node3 corosync[50794]: info    [WD    ] no resources configured.
Mar 20 10:55:12 node3 corosync[50794]: notice  [SERV  ] Service engine loaded: corosync watchdog service [7]
Mar 20 10:55:12 node3 corosync[50794]: notice  [QUORUM] Using quorum provider corosync_votequorum
Mar 20 10:55:12 node3 corosync[50794]: crit    [QUORUM] Quorum provider: corosync_votequorum failed to initialize.
Mar 20 10:55:12 node3 corosync[50794]: error   [SERV  ] Service engine 'corosync_quorum' failed to load for reason 'configuration error: nodelist or quorum.expected_votes must be configured!'
Mar 20 10:55:12 node3 corosync[50794]: error   [MAIN  ] Corosync Cluster Engine exiting with status 20 at service.c:356.
Mar 20 10:55:12 node3 systemd[1]: corosync.service: Main process exited, code=exited, status=20/n/a
Mar 20 10:55:12 node3 systemd[1]: Failed to start Corosync Cluster Engine.
Mar 20 10:55:12 node3 systemd[1]: corosync.service: Unit entered failed state.
Mar 20 10:55:12 node3 systemd[1]: corosync.service: Failed with result 'exit-code'.
...
...

here is my corosync conf file:

logging {
  debug: off
  to_syslog: yes
}

nodelist {
  node {
    name: node3
    nodeid: 1
    quorum_votes: 1
    ring0_addr: node3
  }
  node {
    name: node2
    nodeid: 2
    quorum_votes: 1
    ring0_addr: node2
  }
  node {
    name: node1
    nodeid: 4
    quorum_votes: 1
    ring0_addr: node1
  }
}

quorum {
  provider: corosync_votequorum
}

totem {
  cluster_name: mycluster
  config_version: 27
  interface {
    bindnetaddr: 0.0.0.0
    ringnumber: 0
  }
  ip_version: ipv4
  secauth: on
  version: 2
}

Would someone please tell me what I did wrong ?

Many thanks

 

[Dubard]

Hi Everybody,

I figured out how to solve my problem. Follow the recommendations on this page: https://pve.proxmox.com/wiki/Separate_Cluster_Network
(see "Setup on a running cluster" section).

Thanks