[CAN BE DELETED - ERROR] After upgrade from 6.4 to 7.0, 3 node cluster is not ready (corosync)

J

Javier Estellés Dasí

Member
Feb 10, 2016
24
0
21
58
[SOLVED] After the update the cluster network card was not working properly. Sorry.

Hi!

In a working cluster I have updated to the latest version and since the restart I have no quorum. Here I show you the execution of some commands:

pvecm -status shows this info.

Code: 
Cluster information
-------------------
Name:             46017882CLU
Config Version:   3
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Tue Jul  6 18:13:29 2021
Quorum provider:  corosync_votequorum
Nodes:            1
Node ID:          0x00000001
Ring ID:          1.f8
Quorate:          No

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      1
Quorum:           2 Activity blocked
Flags:        

Membership information
----------------------
    Nodeid      Votes Name
0x00000001          1 10.3.0.240 (local)


Code: 
oot@46017882HV1:~# systemctl status pve-cluster corosync
● pve-cluster.service - The Proxmox VE cluster filesystem
     Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2021-07-06 18:07:27 CEST; 18min ago
    Process: 2953 ExecStart=/usr/bin/pmxcfs (code=exited, status=0/SUCCESS)
   Main PID: 3127 (pmxcfs)
      Tasks: 7 (limit: 38207)
     Memory: 52.3M
        CPU: 1.484s
     CGroup: /system.slice/pve-cluster.service
             └─3127 /usr/bin/pmxcfs

Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [dcdb] crit: cpg_initialize failed: 2
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [dcdb] crit: can't initialize service
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [status] crit: cpg_initialize failed: 2
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [status] crit: can't initialize service
Jul 06 18:07:27 46017882HV1 systemd[1]: Started The Proxmox VE cluster filesystem.
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [status] notice: update cluster info (cluster name  46017882CLU, version = 3)
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [dcdb] notice: members: 1/3127
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [dcdb] notice: all data is up to date
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [status] notice: members: 1/3127
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [status] notice: all data is up to date

● corosync.service - Corosync Cluster Engine
     Loaded: loaded (/lib/systemd/system/corosync.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2021-07-06 18:07:28 CEST; 18min ago
       Docs: man:corosync
             man:corosync.conf
             man:corosync_overview
   Main PID: 3182 (corosync)
      Tasks: 9 (limit: 38207)
     Memory: 148.9M
        CPU: 7.211s
     CGroup: /system.slice/corosync.service
             └─3182 /usr/sbin/corosync -f

Jul 06 18:07:28 46017882HV1 corosync[3182]:   [KNET  ] host: host: 3 (passive) best link: 0 (pri: 1)
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [KNET  ] host: host: 3 has no active links
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [KNET  ] host: host: 3 (passive) best link: 0 (pri: 1)
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [KNET  ] host: host: 3 has no active links
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [QUORUM] Sync members[1]: 1
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [QUORUM] Sync joined[1]: 1
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [TOTEM ] A new membership (1.f8) was formed. Members joined: 1
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [QUORUM] Members[1]: 1
Jul 06 18:07:28 46017882HV1 corosync[3182]:   [MAIN  ] Completed service synchronization, ready to provide service.
Jul 06 18:07:28 46017882HV1 systemd[1]: Started Corosync Cluster Engine.

Code: 
root@46017882HV1:~# journalctl -b -u pve-cluster
-- Journal begins at Tue 2021-07-06 17:11:18 CEST, ends at Tue 2021-07-06 18:29:04 CEST. --
Jul 06 18:07:25 46017882HV1 systemd[1]: Starting The Proxmox VE cluster filesystem...
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [quorum] crit: quorum_initialize failed: 2
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [quorum] crit: can't initialize service
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [confdb] crit: cmap_initialize failed: 2
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [confdb] crit: can't initialize service
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [dcdb] crit: cpg_initialize failed: 2
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [dcdb] crit: can't initialize service
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [status] crit: cpg_initialize failed: 2
Jul 06 18:07:26 46017882HV1 pmxcfs[3127]: [status] crit: can't initialize service
Jul 06 18:07:27 46017882HV1 systemd[1]: Started The Proxmox VE cluster filesystem.
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [status] notice: update cluster info (cluster name  46017882CLU, version = 3)
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [dcdb] notice: members: 1/3127
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [dcdb] notice: all data is up to date
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [status] notice: members: 1/3127
Jul 06 18:07:32 46017882HV1 pmxcfs[3127]: [status] notice: all data is up to date

In node 1 and 3

Code: 
root@46017882HV3:~# systemctl status pve-cluster.service
● pve-cluster.service - The Proxmox VE cluster filesystem
     Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2021-07-06 18:09:44 CEST; 34min ago
    Process: 2613 ExecStart=/usr/bin/pmxcfs (code=exited, status=0/SUCCESS)
   Main PID: 2660 (pmxcfs)
      Tasks: 7 (limit: 38207)
     Memory: 32.7M
        CPU: 1.541s
     CGroup: /system.slice/pve-cluster.service
             └─2660 /usr/bin/pmxcfs

Jul 06 18:09:43 46017882HV3 pmxcfs[2660]: [dcdb] crit: cpg_initialize failed: 2
Jul 06 18:09:43 46017882HV3 pmxcfs[2660]: [dcdb] crit: can't initialize service
Jul 06 18:09:43 46017882HV3 pmxcfs[2660]: [status] crit: cpg_initialize failed: 2
Jul 06 18:09:43 46017882HV3 pmxcfs[2660]: [status] crit: can't initialize service
Jul 06 18:09:44 46017882HV3 systemd[1]: Started The Proxmox VE cluster filesystem.
Jul 06 18:09:49 46017882HV3 pmxcfs[2660]: [status] notice: update cluster info (cluster name  46017882CLU, version = 3)
Jul 06 18:09:49 46017882HV3 pmxcfs[2660]: [dcdb] notice: members: 3/2660
Jul 06 18:09:49 46017882HV3 pmxcfs[2660]: [dcdb] notice: all data is up to date
Jul 06 18:09:49 46017882HV3 pmxcfs[2660]: [status] notice: members: 3/2660
Jul 06 18:09:49 46017882HV3 pmxcfs[2660]: [status] notice: all data is up to date

But at node 2:

Code: 
root@46017882HV2:~# systemctl status pve-cluster.service
● pve-cluster.service - The Proxmox VE cluster filesystem
     Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2021-07-06 18:41:57 CEST; 42s ago
    Process: 22143 ExecStart=/usr/bin/pmxcfs (code=exited, status=0/SUCCESS)
   Main PID: 22144 (pmxcfs)
      Tasks: 5 (limit: 38207)
     Memory: 20.9M
        CPU: 111ms
     CGroup: /system.slice/pve-cluster.service
             └─22144 /usr/bin/pmxcfs

Jul 06 18:41:56 46017882HV2 systemd[1]: Starting The Proxmox VE cluster filesystem...
Jul 06 18:41:56 46017882HV2 pmxcfs[22144]: [status] notice: update cluster info (cluster name  46017882CLU, version = 3)
Jul 06 18:41:56 46017882HV2 pmxcfs[22144]: [dcdb] notice: members: 2/22144
Jul 06 18:41:56 46017882HV2 pmxcfs[22144]: [dcdb] notice: all data is up to date
Jul 06 18:41:56 46017882HV2 pmxcfs[22144]: [status] notice: members: 2/22144
Jul 06 18:41:56 46017882HV2 pmxcfs[22144]: [status] notice: all data is up to date
Jul 06 18:41:57 46017882HV2 systemd[1]: Started The Proxmox VE cluster filesystem.
The last command I have run in case it helps was:

Code: 
root@46017882HV1:~# pmxcfs
[main] notice: unable to acquire pmxcfs lock - trying again

[main] crit: unable to acquire pmxcfs lock: Resource temporarily unavailable
[main] notice: exit proxmox configuration filesystem (-1)

Can someone give me some help.

Thanks in advance
 
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom