after power lost -> TASK ERROR: cluster not ready - no quorum?

I

informant

Renowned Member
Jan 31, 2012
793
10
83
hi, after power lost on cluster i have following problem on cluster: TASK ERROR: cluster not ready - no quorum?
what can i do here, after reboot its the same issue.

pvecm status
Cannot initialize CMAP service

manual start show the same error.

what can i do here, plwase help, thanks and best regards.
last version of proxmox 4.* is installed.

the cluster works fine before, since more as 4 years...
the error comes on all startet nodes, why? works before without problems. what can i do here?

systemctl status corosync.service
* corosync.service - Corosync Cluster Engine
Loaded: loaded (/lib/systemd/system/corosync.service; enabled)
Active: failed (Result: exit-code) since Fr 2016-06-17 15:09:30 CEST; 7min ago
Process: 4727 ExecStart=/usr/share/corosync/corosync start (code=exited, status=1/FAILURE)

Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Unloading all Corosync service engines.
Jun 17 15:08:29 pegasus corosync[4738]: [QB ] withdrawing server sockets
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Service engine unloaded: corosync vote quorum service v1.0
Jun 17 15:08:29 pegasus corosync[4738]: [QB ] withdrawing server sockets
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Service engine unloaded: corosync configuration map access
Jun 17 15:08:29 pegasus corosync[4738]: [QB ] withdrawing server sockets
Jun 17 15:09:30 pegasus corosync[4727]: Starting Corosync Cluster Engine (corosync): [FAILED]
Jun 17 15:09:30 pegasus systemd[1]: corosync.service: control process exited, code=exited status=1
Jun 17 15:09:30 pegasus systemd[1]: Failed to start Corosync Cluster Engine.
Jun 17 15:09:30 pegasus systemd[1]: Unit corosync.service entered failed state.
Click to expand...

journalctl -xn
-- Logs begin at Fr 2016-06-17 15:02:27 CEST, end at Fr 2016-06-17 15:17:22 CEST. --
Jun 17 15:17:10 pegasus pmxcfs[4717]: [dcdb] crit: cpg_initialize failed: 2
Jun 17 15:17:10 pegasus pmxcfs[4717]: [status] crit: cpg_initialize failed: 2
Jun 17 15:17:16 pegasus pmxcfs[4717]: [quorum] crit: quorum_initialize failed: 2
Jun 17 15:17:16 pegasus pmxcfs[4717]: [confdb] crit: cmap_initialize failed: 2
Jun 17 15:17:16 pegasus pmxcfs[4717]: [dcdb] crit: cpg_initialize failed: 2
Jun 17 15:17:16 pegasus pmxcfs[4717]: [status] crit: cpg_initialize failed: 2
Jun 17 15:17:22 pegasus pmxcfs[4717]: [quorum] crit: quorum_initialize failed: 2
Jun 17 15:17:22 pegasus pmxcfs[4717]: [confdb] crit: cmap_initialize failed: 2
Jun 17 15:17:22 pegasus pmxcfs[4717]: [dcdb] crit: cpg_initialize failed: 2
Jun 17 15:17:22 pegasus pmxcfs[4717]: [status] crit: cpg_initialize failed: 2
Click to expand...

thanks and regards
 
Last edited:
Jun 17 15:08:27 pegasus pmxcfs[1202]: [main] notice: exit proxmox configuration filesystem (0)
Jun 17 15:08:27 pegasus systemd[1]: Failed to reset devices.list on /system.slice: Invalid argument
Jun 17 15:08:27 pegasus pveproxy[1653]: ipcc_send_rec failed: Verbindungsaufbau abgelehnt
Jun 17 15:08:27 pegasus pveproxy[1653]: ipcc_send_rec failed: Verbindungsaufbau abgelehnt
Jun 17 15:08:27 pegasus pveproxy[1653]: ipcc_send_rec failed: Verbindungsaufbau abgelehnt
Jun 17 15:08:27 pegasus pveproxy[1653]: ipcc_send_rec failed: Verbindungsaufbau abgelehnt
Jun 17 15:08:28 pegasus pmxcfs[4717]: [quorum] crit: quorum_initialize failed: 2
Jun 17 15:08:28 pegasus pmxcfs[4717]: [quorum] crit: can't initialize service
Jun 17 15:08:28 pegasus pmxcfs[4717]: [confdb] crit: cmap_initialize failed: 2
Jun 17 15:08:28 pegasus pmxcfs[4717]: [confdb] crit: can't initialize service
Jun 17 15:08:28 pegasus pmxcfs[4717]: [dcdb] crit: cpg_initialize failed: 2
Jun 17 15:08:28 pegasus pmxcfs[4717]: [dcdb] crit: can't initialize service
Jun 17 15:08:28 pegasus pmxcfs[4717]: [status] crit: cpg_initialize failed: 2
Jun 17 15:08:28 pegasus pmxcfs[4717]: [status] crit: can't initialize service
Jun 17 15:08:28 pegasus pveproxy[1653]: ipcc_send_rec failed: Verbindungsaufbau abgelehnt
Jun 17 15:08:28 pegasus pveproxy[1653]: ipcc_send_rec failed: Verbindungsaufbau abgelehnt
Jun 17 15:08:28 pegasus pveproxy[1653]: ipcc_send_rec failed: Verbindungsaufbau abgelehnt
Jun 17 15:08:28 pegasus pve-ha-crm[1647]: ipcc_send_rec failed: Der Socket ist nicht verbunden
Jun 17 15:08:28 pegasus pve-ha-crm[1647]: ipcc_send_rec failed: Verbindungsaufbau abgelehnt
Jun 17 15:08:28 pegasus pve-ha-crm[1647]: ipcc_send_rec failed: Verbindungsaufbau abgelehnt
Jun 17 15:08:28 pegasus pveproxy[1653]: ipcc_send_rec failed: Verbindungsaufbau abgelehnt
Jun 17 15:08:28 pegasus pveproxy[1653]: ipcc_send_rec failed: Verbindungsaufbau abgelehnt
Jun 17 15:08:28 pegasus pveproxy[1653]: ipcc_send_rec failed: Verbindungsaufbau abgelehnt
Jun 17 15:08:29 pegasus corosync[4733]: [MAIN ] Corosync Cluster Engine ('2.3.5.15-e2b6b'): started and ready to provide service.
Jun 17 15:08:29 pegasus corosync[4733]: [MAIN ] Corosync built-in features: augeas systemd pie relro bindnow
Jun 17 15:08:29 pegasus pveproxy[1651]: ipcc_send_rec failed: Der Socket ist nicht verbunden
Jun 17 15:08:29 pegasus pvedaemon[1640]: ipcc_send_rec failed: Der Socket ist nicht verbunden
Jun 17 15:08:29 pegasus corosync[4738]: [TOTEM ] Initializing transport (UDP/IP Multicast).
Jun 17 15:08:29 pegasus corosync[4738]: [TOTEM ] Initializing transmit/receive security (NSS) crypto: aes256 hash: sha1
Jun 17 15:08:29 pegasus corosync[4738]: [TOTEM ] The network interface [217.69.254.67] is now up.
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Service engine loaded: corosync configuration map access [0]
Jun 17 15:08:29 pegasus corosync[4738]: [QB ] server name: cmap
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Service engine loaded: corosync configuration service [1]
Jun 17 15:08:29 pegasus corosync[4738]: [QB ] server name: cfg
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Service engine loaded: corosync cluster closed process group service v1.01 [2]
Jun 17 15:08:29 pegasus corosync[4738]: [QB ] server name: cpg
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Service engine loaded: corosync profile loading service [4]
Jun 17 15:08:29 pegasus corosync[4738]: [QUORUM] Using quorum provider corosync_votequorum
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Service engine loaded: corosync vote quorum service v1.0 [5]
Jun 17 15:08:29 pegasus corosync[4738]: [QB ] server name: votequorum
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Service engine loaded: corosync cluster quorum service v0.1 [3]
Jun 17 15:08:29 pegasus corosync[4738]: [QB ] server name: quorum
Jun 17 15:08:29 pegasus corosync[4738]: [TOTEM ] A new membership (217.69.254.67:26588) was formed. Members joined: 1
Jun 17 15:08:29 pegasus corosync[4738]: [QUORUM] Members[1]: 1
Jun 17 15:08:29 pegasus corosync[4738]: [MAIN ] Completed service synchronization, ready to provide service.
Jun 17 15:08:29 pegasus corosync[4738]: [TOTEM ] A new membership (217.69.254.64:26592) was formed. Members joined: 3 6 5
Jun 17 15:08:29 pegasus corosync[4738]: [CMAP ] Received config version (13) is different than my config version (14)! Exiting
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Unloading all Corosync service engines.
Jun 17 15:08:29 pegasus corosync[4738]: [QB ] withdrawing server sockets
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Service engine unloaded: corosync vote quorum service v1.0
Jun 17 15:08:29 pegasus corosync[4738]: [QB ] withdrawing server sockets
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Service engine unloaded: corosync configuration map access
Jun 17 15:08:29 pegasus corosync[4738]: [QB ] withdrawing server sockets
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Service engine unloaded: corosync configuration service
Jun 17 15:08:29 pegasus corosync[4738]: [QB ] withdrawing server sockets
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Service engine unloaded: corosync cluster closed process group service v1.01
Jun 17 15:08:29 pegasus corosync[4738]: [QB ] withdrawing server sockets
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Service engine unloaded: corosync cluster quorum service v0.1
Jun 17 15:08:29 pegasus corosync[4738]: [SERV ] Service engine unloaded: corosync profile loading service
Jun 17 15:08:29 pegasus corosync[4738]: [MAIN ] Corosync Cluster Engine exiting normally
Jun 17 15:08:29 pegasus pve-ha-lrm[1657]: ipcc_send_rec failed: Der Socket ist nicht verbunden
Jun 17 15:08:34 pegasus pmxcfs[4717]: [quorum] crit: quorum_initialize failed: 2
...
Jun 17 15:08:46 pegasus pmxcfs[4717]: [confdb] crit: cmap_initialize failed: 2
Jun 17 15:08:46 pegasus pmxcfs[4717]: [dcdb] crit: cpg_initialize failed: 2
Jun 17 15:08:46 pegasus pmxcfs[4717]: [status] crit: cpg_initialize failed: 2
Jun 17 15:08:49 pegasus systemd-sysv-generator[9854]: Ignoring creation of an alias umountiscsi.service for itself
Jun 17 15:08:49 pegasus systemd[1]: [/run/systemd/generator.late/rgmanager.service:7] Failed to add dependency on +vz.service, ignoring: Invalid argument
Jun 17 15:08:49 pegasus systemd[1]: [/run/systemd/generator.late/rgmanager.service:7] Failed to add dependency on +qemu-server.service, ignoring: Invalid argument
Jun 17 15:08:49 pegasus systemd[1]: [/run/systemd/generator.late/qemu-server.service:7] Failed to add dependency on +iscsi.service, ignoring: Invalid argument
Jun 17 15:08:52 pegasus pmxcfs[4717]: [quorum] crit: quorum_initialize failed: 2
Jun 17 15:08:52 pegasus pmxcfs[4717]: [confdb] crit: cmap_initialize failed: 2
Jun 17 15:08:52 pegasus pmxcfs[4717]: [dcdb] crit: cpg_initialize failed: 2
Jun 17 15:08:52 pegasus pmxcfs[4717]: [status] crit: cpg_initialize failed: 2
Jun 17 15:08:58 pegasus pmxcfs[4717]: [quorum] crit: quorum_initialize failed: 2
Jun 17 15:08:58 pegasus pmxcfs[4717]: [confdb] crit: cmap_initialize failed: 2
Jun 17 15:08:58 pegasus pmxcfs[4717]: [dcdb] crit: cpg_initialize failed: 2
Jun 17 15:08:58 pegasus pmxcfs[4717]: [status] crit: cpg_initialize failed: 2
Jun 17 15:09:01 pegasus cron[1283]: (*system*pveupdate) RELOAD (/etc/cron.d/pveupdate)
Jun 17 15:09:04 pegasus pmxcfs[4717]: [quorum] crit: quorum_initialize failed: 2
Click to expand...
 
Jun 17 15:09:28 pegasus pmxcfs[4717]: [quorum] crit: quorum_initialize failed: 2
Jun 17 15:09:28 pegasus pmxcfs[4717]: [confdb] crit: cmap_initialize failed: 2
Jun 17 15:09:28 pegasus pmxcfs[4717]: [dcdb] crit: cpg_initialize failed: 2
Jun 17 15:09:28 pegasus pmxcfs[4717]: [status] crit: cpg_initialize failed: 2
Jun 17 15:09:30 pegasus corosync[4727]: Starting Corosync Cluster Engine (corosync): [FAILED]
Jun 17 15:09:30 pegasus systemd[1]: corosync.service: control process exited, code=exited status=1
Jun 17 15:09:30 pegasus systemd[1]: Failed to start Corosync Cluster Engine.
Jun 17 15:09:30 pegasus systemd[1]: Unit corosync.service entered failed state.
Jun 17 15:09:30 pegasus systemd[1]: Failed to reset devices.list on /system.slice: Invalid argument
Jun 17 15:09:32 pegasus pvedaemon[10409]: send HUP to 1639
Jun 17 15:09:32 pegasus pvedaemon[1639]: received signal HUP
Jun 17 15:09:32 pegasus pvedaemon[1639]: server closing
Jun 17 15:09:32 pegasus pvedaemon[1639]: server shutdown (restart)
Jun 17 15:09:32 pegasus pvedaemon[1640]: worker exit
Jun 17 15:09:33 pegasus pvedaemon[10428]: worker exit
Jun 17 15:09:34 pegasus pmxcfs[4717]: [quorum] crit: quorum_initialize failed: 2
Jun 17 15:09:34 pegasus pmxcfs[4717]: [confdb] crit: cmap_initialize failed: 2
Jun 17 15:09:34 pegasus pmxcfs[4717]: [dcdb] crit: cpg_initialize failed: 2
Jun 17 15:09:34 pegasus pmxcfs[4717]: [status] crit: cpg_initialize failed: 2
Jun 17 15:09:34 pegasus pvedaemon[1639]: restarting server
Jun 17 15:09:34 pegasus pvedaemon[1639]: worker 1640 finished
Jun 17 15:09:34 pegasus pvedaemon[1639]: worker 1642 finished
Jun 17 15:09:34 pegasus pvedaemon[1639]: worker 1641 finished
Jun 17 15:09:34 pegasus pvedaemon[1639]: starting 3 worker(s)
Jun 17 15:09:34 pegasus pvedaemon[1639]: worker 10450 started
Jun 17 15:09:34 pegasus pvedaemon[1639]: worker 10451 started
Jun 17 15:09:34 pegasus pvedaemon[1639]: worker 10452 started
Jun 17 15:09:34 pegasus pveproxy[10431]: send HUP to 1650
Jun 17 15:09:34 pegasus pveproxy[1650]: received signal HUP
Jun 17 15:09:34 pegasus pveproxy[1650]: server closing
Jun 17 15:09:34 pegasus pveproxy[1650]: server shutdown (restart)
Jun 17 15:09:34 pegasus pveproxy[1653]: worker exit
Jun 17 15:09:34 pegasus pveproxy[1651]: worker exit
Jun 17 15:09:35 pegasus pveproxy[10453]: worker exit
Jun 17 15:09:36 pegasus spiceproxy[10456]: send HUP to 1658
Jun 17 15:09:36 pegasus spiceproxy[1658]: received signal HUP
Jun 17 15:09:36 pegasus spiceproxy[1658]: server closing
Jun 17 15:09:36 pegasus spiceproxy[1658]: server shutdown (restart)
Jun 17 15:09:36 pegasus spiceproxy[1659]: worker exit
Jun 17 15:09:36 pegasus pveproxy[1650]: restarting server
Jun 17 15:09:36 pegasus pveproxy[1650]: worker 1652 finished
Jun 17 15:09:36 pegasus pveproxy[1650]: worker 1651 finished
Jun 17 15:09:36 pegasus pveproxy[1650]: worker 1653 finished
Jun 17 15:09:36 pegasus pveproxy[1650]: starting 3 worker(s)
Jun 17 15:09:36 pegasus pveproxy[1650]: worker 10460 started
Jun 17 15:09:36 pegasus pveproxy[1650]: worker 10461 started
Jun 17 15:09:36 pegasus pveproxy[1650]: worker 10463 started
Jun 17 15:09:38 pegasus pvestatd[10462]: send HUP to 1313
Jun 17 15:09:38 pegasus pvestatd[1313]: received signal HUP
Jun 17 15:09:38 pegasus pvestatd[1313]: server shutdown (restart)
Jun 17 15:09:38 pegasus spiceproxy[1658]: restarting server
Jun 17 15:09:38 pegasus spiceproxy[1658]: worker 1659 finished
Jun 17 15:09:38 pegasus spiceproxy[1658]: starting 1 worker(s)
Jun 17 15:09:38 pegasus spiceproxy[1658]: worker 10506 started
Jun 17 15:09:39 pegasus pvestatd[1313]: restarting server
Jun 17 15:09:40 pegasus pmxcfs[4717]: [quorum] crit: quorum_initialize failed: 2
Jun 17 15:09:40 pegasus pmxcfs[4717]: [confdb] crit: cmap_initialize failed: 2
Jun 17 15:09:40 pegasus pmxcfs[4717]: [dcdb] crit: cpg_initialize failed: 2
Jun 17 15:09:40 pegasus pmxcfs[4717]: [status] crit: cpg_initialize failed: 2
Jun 17 15:09:46 pegasus pmxcfs[4717]: [quorum] crit: quorum_initialize failed: 2
Jun 17 15:09:46 pegasus pmxcfs[4717]: [confdb] crit: cmap_initialize failed: 2
Jun 17 15:09:46 pegasus pmxcfs[4717]: [dcdb] crit: cpg_initialize failed: 2
...
Jun 17 15:10:22 pegasus pmxcfs[4717]: [status] crit: cpg_initialize failed: 2
Jun 17 15:10:28 pegasus pmxcfs[4717]: [quorum] crit: quorum_initialize failed: 2
Jun 17 15:10:28 pegasus pmxcfs[4717]: [confdb] crit: cmap_initialize failed: 2
Jun 17 15:10:28 pegasus pmxcfs[4717]: [dcdb] crit: cpg_initialize failed: 2
Jun 17 15:10:28 pegasus pmxcfs[4717]: [status] crit: cpg_initialize failed: 2
Jun 17 15:10:28 pegasus systemd-timesyncd[485]: interval/delta/delay/jitter/drift 512s/+0.001s/0.034s/0.002s/-15ppm (ignored)
...
Jun 17 15:12:10 pegasus pmxcfs[4717]: [quorum] crit: quorum_initialize failed: 2
Jun 17 15:12:10 pegasus pmxcfs[4717]: [confdb] crit: cmap_initialize failed: 2
Jun 17 15:12:10 pegasus pmxcfs[4717]: [dcdb] crit: cpg_initialize failed: 2
Jun 17 15:12:10 pegasus pmxcfs[4717]: [status] crit: cpg_initialize failed: 2
Click to expand...
 
hi, a info, i can not start vms on the nodes and on cluster if they have this error since restart - what can i do? please help :( i have no more ideas...
 
I have the same issue and it has left me in a serious pickle. Help needed!
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom