[SOLVED] My Proxmox don't work after a failled cluster

[emmanu7474]

Hello guys,

I'm new on this forum, I was used Proxmox since many years, no problems with that ! But at this time... I've a big problem...
Before, I configured a cluster, it was worked very fine, and it was failled one time...
My master works fine ; I use also one pve.
But I tried to fix my cluster, and after a reboot : Impossible to see the gui.

root@proxmox01:~# pveversion -v
proxmox-ve: 7.4-1 (running kernel: 5.15.107-1-pve)
pve-manager: 7.4-3 (running version: 7.4-3/9002ab8a)
pve-kernel-5.15: 7.4-2
pve-kernel-5.13: 7.1-9
pve-kernel-5.4: 6.4-11
pve-kernel-5.15.107-1-pve: 5.15.107-1
pve-kernel-5.15.104-1-pve: 5.15.104-2
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.4.157-1-pve: 5.4.157-1
pve-kernel-5.4.34-1-pve: 5.4.34-2
ceph-fuse: 14.2.21-1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libproxmox-rs-perl: 0.2.1
libpve-access-control: 7.4-2
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.3-4
libpve-guest-common-perl: 4.2-4
libpve-http-server-perl: 4.2-3
libpve-rs-perl: 0.7.5
libpve-storage-perl: 7.4-2
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
proxmox-backup-client: 2.4.1-1
proxmox-backup-file-restore: 2.4.1-1
proxmox-kernel-helper: 7.4-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.1-1
proxmox-widget-toolkit: 3.6.5
pve-cluster: 7.3-3
pve-container: 4.4-3
pve-docs: 7.4-2
pve-edk2-firmware: 3.20230228-2
pve-firewall: 4.3-1
pve-firmware: 3.6-5
pve-ha-manager: 3.6.1
pve-i18n: 2.12-1
pve-qemu-kvm: 7.2.0-8
pve-xtermjs: 4.16.0-1
qemu-server: 7.4-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.11-pve1

My corosync.conf :

root@proxmox01:~# cat /etc/pve/corosync.conf
logging {
  debug: off
  to_syslog: yes
}

nodelist {
  node {
    name: proxmox01
    nodeid: 2
    quorum_votes: 1
    ring0_addr: 192.168.10.12
  }
}

quorum {
  expected_votes: 1
  provider: corosync_votequorum
  two_node: 1
}

totem {
  cluster_name: Cluster2
  config_version: 3
  interface {
    linknumber: 0
  }
  ip_version: ipv4-6
  secauth: on
  version: 2
}

Logs :

corosync.service - Corosync Cluster Engine
     Loaded: loaded (/lib/systemd/system/corosync.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2023-05-04 21:41:55 CEST; 23s ago
       Docs: man:corosync
             man:corosync.conf
             man:corosync_overview
    Process: 39004 ExecStart=/usr/sbin/corosync -f $COROSYNC_OPTIONS (code=exited, status=8)
   Main PID: 39004 (code=exited, status=8)
        CPU: 6ms

May 04 21:41:55 proxmox01.mydomain.fr systemd[1]: Starting Corosync Cluster Engine...
May 04 21:41:55 proxmox01.mydomain.fr corosync[39004]:   [MAIN  ] Corosync Cluster Engine 3.1.7 starting up
May 04 21:41:55 proxmox01.mydomain.fr corosync[39004]:   [MAIN  ] Corosync built-in features: dbus monitoring watchdog s>
May 04 21:41:55 proxmox01.mydomain.fr corosync[39004]:   [MAIN  ] Could not open /etc/corosync/authkey: No such file or >
May 04 21:41:55 proxmox01.mydomain.fr corosync[39004]:   [MAIN  ] Corosync Cluster Engine exiting with status 8 at main.>
May 04 21:41:55 proxmox01.mydomain.fr systemd[1]: corosync.service: Main process exited, code=exited, status=8/n/a
May 04 21:41:55 proxmox01.mydomain.fr systemd[1]: corosync.service: Failed with result 'exit-code'.
May 04 21:41:55 proxmox01.mydomain.fr systemd[1]: Failed to start Corosync Cluster Engine.

May 04 21:45:01 proxmox01.mydomain.fr cron[6320]: (*system*vzdump) CAN'T OPEN SYMLINK (/etc/cron.d/vzdump)
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[39138]: worker exit
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[6611]: worker 39138 finished
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[6611]: starting 1 worker(s)
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[6611]: worker 39147 started
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[39147]: /etc/pve/local/pve-ssl.key: failed to load local private key (key>
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[39139]: worker exit
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[6611]: worker 39139 finished
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[6611]: starting 1 worker(s)
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[6611]: worker 39148 started
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[39148]: /etc/pve/local/pve-ssl.key: failed to load local private key (key>
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[39140]: worker exit
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[6611]: worker 39140 finished
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[6611]: starting 1 worker(s)
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[6611]: worker 39149 started
May 04 21:45:02 proxmox01.mydomain.fr pveproxy[39149]: /etc/pve/local/pve-ssl.key: failed to load local private key (key>
lines 1036-1053/1053 (END)

Thank you to help me !

I don't see how can I fix my cluster...
Regards

And when I ask a status, I had this one :

root@proxmox01:~# pvecm status
ipcc_send_rec[1] failed: Connection refused
ipcc_send_rec[2] failed: Connection refused
ipcc_send_rec[3] failed: Connection refused
Unable to load access control list: Connection refused

So, it is solved now ! )
Thank you very much to this post : https://forum.proxmox.com/threads/proxmox-ve-6-removing-cluster-configuration.56259/#post-370010

And congrutulations to Proxmox and at this forum !

;o)

 

[gurubert]

Could not open /etc/corosync/authkey

It looks like the node misses a vital file. Do you have a backup?
You could try and copy that file from another node.

If there is no authkey on any node you could generate a new one and distribute it to each node: https://manpages.debian.org/bullseye/corosync/corosync-keygen.8.en.html