[CAN BE DELETED - ERROR] After upgrade from 6.4 to 7.0, 3 node cluster is not ready (corosync)

J

Javier Estellés Dasí

Member
Feb 10, 2016
24
0
21
59
[SOLVED] After the update the cluster network card was not working properly. Sorry.

Hi!

In a working cluster I have updated to the latest version and since the restart I have no quorum. Here I show you the execution of some commands:

pvecm -status shows this info.

Code: 
Cluster information
-------------------
Name:             46017882CLU
Config Version:   3
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Tue Jul  6 18:13:29 2021
Quorum provider:  corosync_votequorum
Nodes:            1
Node ID:          0x00000001
Ring ID:          1.f8
Quorate:          No

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      1
Quorum:           2 Activity blocked
Flags:        

Membership information
----------------------
    Nodeid      Votes Name
0x00000001          1 10.3.0.240 (local)


Code: 
oot@46017882HV1:~# systemctl status pve-cluster corosync
● pve-cluster.service - The Proxmox VE cluster filesystem
     Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2021-07-06 18:07:27 CEST; 18min ago
    Process: 2953 ExecStart=/usr/bin/pmxcfs (code=exited, status=0/SUCCESS)
   Main PID: 3127 (pmxcfs)
      Tasks: 7 (limit: 38207)
     Memory: 52.3M
        CPU: 1.484s
     CGroup: /system.slice/pve-cluster.service
             └─3127 /usr/bin/pmxcfs

Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [dcdb] crit: cpg_initialize failed: 2
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [dcdb] crit: can't initialize service
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [status] crit: cpg_initialize failed: 2
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [status] crit: can't initialize service
Jul 06 18:07:27 46017882HV1 systemd[1]: Started The Proxmox VE cluster filesystem.
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [status] notice: update cluster info (cluster name  46017882CLU, version = 3)
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [dcdb] notice: members: 1/3127
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [dcdb] notice: all data is up to date
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [status] notice: members: 1/3127
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [status] notice: all data is up to date

● corosync.service - Corosync Cluster Engine
     Loaded: loaded (/lib/systemd/system/corosync.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2021-07-06 18:07:28 CEST; 18min ago
       Docs: man:corosync
             man:corosync.conf
             man:corosync_overview
   Main PID: 3182 (corosync)
      Tasks: 9 (limit: 38207)
     Memory: 148.9M
        CPU: 7.211s
     CGroup: /system.slice/corosync.service
             └─3182 /usr/sbin/corosync -f

Jul 06 18:07:28 46017882HV1 corosync[3182]:   [KNET  ] host: host: 3 (passive) best link: 0 (pri: 1)
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [KNET  ] host: host: 3 has no active links
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [KNET  ] host: host: 3 (passive) best link: 0 (pri: 1)
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [KNET  ] host: host: 3 has no active links
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [QUORUM] Sync members[1]: 1
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [QUORUM] Sync joined[1]: 1
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [TOTEM ] A new membership (1.f8) was formed. Members joined: 1
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [QUORUM] Members[1]: 1
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [MAIN  ] Completed service synchronization, ready to provide service.
Jul 06 18:07:28 46017882HV1 systemd[1]: Started Corosync Cluster Engine.

Code: 
root@46017882HV1:~# journalctl -b -u pve-cluster
-- Journal begins at Tue 2021-07-06 17:11:18 CEST, ends at Tue 2021-07-06 18:29:04 CEST. --
Jul 06 18:07:25 46017882HV1 systemd[1]: Starting The Proxmox VE cluster filesystem...
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [quorum] crit: quorum_initialize failed: 2
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [quorum] crit: can't initialize service
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [confdb] crit: cmap_initialize failed: 2
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [confdb] crit: can't initialize service
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [dcdb] crit: cpg_initialize failed: 2
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [dcdb] crit: can't initialize service
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [status] crit: cpg_initialize failed: 2
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [status] crit: can't initialize service
Jul 06 18:07:27 46017882HV1 systemd[1]: Started The Proxmox VE cluster filesystem.
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [status] notice: update cluster info (cluster name  46017882CLU, version = 3)
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [dcdb] notice: members: 1/3127
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [dcdb] notice: all data is up to date
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [status] notice: members: 1/3127
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [status] notice: all data is up to date

In node 1 and 3

Code: 
root@46017882HV3:~# systemctl status pve-cluster.service
● pve-cluster.service - The Proxmox VE cluster filesystem
     Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2021-07-06 18:09:44 CEST; 34min ago
    Process: 2613 ExecStart=/usr/bin/pmxcfs (code=exited, status=0/SUCCESS)
   Main PID: 2660 (pmxcfs)
      Tasks: 7 (limit: 38207)
     Memory: 32.7M
        CPU: 1.541s
     CGroup: /system.slice/pve-cluster.service
             └─2660 /usr/bin/pmxcfs

Jul 06 18:09:43 46017882HV3 pmxcfs[2660]: [dcdb] crit: cpg_initialize failed: 2
Jul 06 18:09:43 46017882HV3 pmxcfs[2660]: [dcdb] crit: can't initialize service
Jul 06 18:09:43 46017882HV3 pmxcfs[2660]: [status] crit: cpg_initialize failed: 2
Jul 06 18:09:43 46017882HV3 pmxcfs[2660]: [status] crit: can't initialize service
Jul 06 18:09:44 46017882HV3 systemd[1]: Started The Proxmox VE cluster filesystem.
Jul 06 18:09:49 46017882HV3 pmxcfs[2660]: [status] notice: update cluster info (cluster name  46017882CLU, version = 3)
Jul 06 18:09:49 46017882HV3 pmxcfs[2660]: [dcdb] notice: members: 3/2660
Jul 06 18:09:49 46017882HV3 pmxcfs[2660]: [dcdb] notice: all data is up to date
Jul 06 18:09:49 46017882HV3 pmxcfs[2660]: [status] notice: members: 3/2660
Jul 06 18:09:49 46017882HV3 pmxcfs[2660]: [status] notice: all data is up to date

But at node 2:

Code: 
root@46017882HV2:~# systemctl status pve-cluster.service
● pve-cluster.service - The Proxmox VE cluster filesystem
     Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2021-07-06 18:41:57 CEST; 42s ago
    Process: 22143 ExecStart=/usr/bin/pmxcfs (code=exited, status=0/SUCCESS)
   Main PID: 22144 (pmxcfs)
      Tasks: 5 (limit: 38207)
     Memory: 20.9M
        CPU: 111ms
     CGroup: /system.slice/pve-cluster.service
             └─22144 /usr/bin/pmxcfs

Jul 06 18:41:56 46017882HV2 systemd[1]: Starting The Proxmox VE cluster filesystem...
Jul 06 18:41:56 46017882HV2 pmxcfs[22144]: [status] notice: update cluster info (cluster name  46017882CLU, version = 3)
Jul 06 18:41:56 46017882HV2 pmxcfs[22144]: [dcdb] notice: members: 2/22144
Jul 06 18:41:56 46017882HV2 pmxcfs[22144]: [dcdb] notice: all data is up to date
Jul 06 18:41:56 46017882HV2 pmxcfs[22144]: [status] notice: members: 2/22144
Jul 06 18:41:56 46017882HV2 pmxcfs[22144]: [status] notice: all data is up to date
Jul 06 18:41:57 46017882HV2 systemd[1]: Started The Proxmox VE cluster filesystem.
The last command I have run in case it helps was:

Code: 
root@46017882HV1:~# pmxcfs
[main] notice: unable to acquire pmxcfs lock - trying again

[main] crit: unable to acquire pmxcfs lock: Resource temporarily unavailable
[main] notice: exit proxmox configuration filesystem (-1)

Can someone give me some help.

Thanks in advance
 
Last edited:
You must log in or register to reply here.
Top Bottom
Back