[SOLVED] Clusterjoin funktioniert nicht

B

BerndKleinert

Active Member
Feb 14, 2020
50
4
28
43
Hallo,

ich habe 2 Proxmox VE 7.2-7 als funktionierenden Cluster und möchte einen weiteren Node hinzufügen. Installation unter Debian 11 habe ich nach der Anleitung:

https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_11_Bullseye

gemacht. Dh. ich habe auf allen 3 Servern in der /etc/hosts die 3 statischen IP's und deren Hostnamen stehen. Außerdem ist Ipv6 überall deaktiviert.

Das schlägt aber immer wieder fehl:

Code: 
root@server-3 ~ # pvecm add 1.1.1.1
Please enter superuser (root) password for '1.1.1.1': ********************
Establishing API connection with host '1.1.1.1'
The authenticity of host '1.1.1.1' can't be established.
X509 SHA256 key fingerprint is 60:46:B9:FE:FF:95:8F:88:F1:E4:34:GG:JJ:II:77:17:BC:F5:7F:06:B4:B7:6A:7E:32:71:8F:3B:FF:FA:2C:BE.
Are you sure you want to continue connecting (yes/no)? yes
Login succeeded.
check cluster join API version
No cluster network links passed explicitly, fallback to local node IP '3.3.3.3'
Request addition of this node
Join request OK, finishing setup locally
stopping pve-cluster service
backup old database to '/var/lib/pve-cluster/backup/config-1661936390.sql.gz'
waiting for quorum...

In der syslog steht:

Code: 
Aug 31 11:40:38 server-3 pvecm[1858]: <root@pam> starting task UPID:server-3:00000743:00003242:630F2C96:clusterjoin::root@pam:
Aug 31 11:40:45 server-3 systemd[1]: Stopping The Proxmox VE cluster filesystem...
Aug 31 11:40:45 server-3 pmxcfs[934]: [main] notice: teardown filesystem
Aug 31 11:40:45 server-3 systemd[1624]: etc-pve.mount: Succeeded.
Aug 31 11:40:45 server-3 systemd[1]: etc-pve.mount: Succeeded.
Aug 31 11:40:45 server-3 pmxcfs[934]: [main] notice: exit proxmox configuration filesystem (0)
Aug 31 11:40:45 server-3 systemd[1]: pve-cluster.service: Succeeded.
Aug 31 11:40:45 server-3 systemd[1]: Stopped The Proxmox VE cluster filesystem.
Aug 31 11:40:45 server-3 systemd[1]: Starting Corosync Cluster Engine...
Aug 31 11:40:45 server-3 systemd[1]: Starting The Proxmox VE cluster filesystem...
Aug 31 11:40:45 server-3 corosync[1868]:   [MAIN  ] Corosync Cluster Engine 3.1.5 starting up
Aug 31 11:40:45 server-3 corosync[1868]:   [MAIN  ] Corosync built-in features: dbus monitoring watchdog systemd xmlconf vqsim nozzle snmp pie relro bindnow
Aug 31 11:40:45 server-3 corosync[1868]:   [TOTEM ] Initializing transport (Kronosnet).
Aug 31 11:40:45 server-3 pmxcfs[1870]: [quorum] crit: quorum_initialize failed: 2
Aug 31 11:40:45 server-3 pmxcfs[1870]: [quorum] crit: can't initialize service
Aug 31 11:40:45 server-3 pmxcfs[1870]: [confdb] crit: cmap_initialize failed: 2
Aug 31 11:40:45 server-3 pmxcfs[1870]: [confdb] crit: can't initialize service
Aug 31 11:40:45 server-3 pmxcfs[1870]: [dcdb] crit: cpg_initialize failed: 2
Aug 31 11:40:45 server-3 pmxcfs[1870]: [dcdb] crit: can't initialize service
Aug 31 11:40:45 server-3 pmxcfs[1870]: [status] crit: cpg_initialize failed: 2
Aug 31 11:40:45 server-3 pmxcfs[1870]: [status] crit: can't initialize service
Aug 31 11:40:45 server-3 kernel: [  136.550998] sctp: Hash tables configured (bind 2048/2048)
Aug 31 11:40:46 server-3 corosync[1868]:   [TOTEM ] totemknet initialized
Aug 31 11:40:46 server-3 corosync[1868]:   [KNET  ] common: crypto_nss.so has been loaded from /usr/lib/x86_64-linux-gnu/kronosnet/crypto_nss.so
Aug 31 11:40:46 server-3 corosync[1868]:   [SERV  ] Service engine loaded: corosync configuration map access [0]
Aug 31 11:40:46 server-3 corosync[1868]:   [QB    ] server name: cmap
Aug 31 11:40:46 server-3 corosync[1868]:   [SERV  ] Service engine loaded: corosync configuration service [1]
Aug 31 11:40:46 server-3 corosync[1868]:   [QB    ] server name: cfg
Aug 31 11:40:46 server-3 corosync[1868]:   [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01 [2]
Aug 31 11:40:46 server-3 corosync[1868]:   [QB    ] server name: cpg
Aug 31 11:40:46 server-3 corosync[1868]:   [SERV  ] Service engine loaded: corosync profile loading service [4]
Aug 31 11:40:46 server-3 corosync[1868]:   [SERV  ] Service engine loaded: corosync resource monitoring service [6]
Aug 31 11:40:46 server-3 corosync[1868]:   [WD    ] Watchdog not enabled by configuration
Aug 31 11:40:46 server-3 corosync[1868]:   [WD    ] resource load_15min missing a recovery key.
Aug 31 11:40:46 server-3 corosync[1868]:   [WD    ] resource memory_used missing a recovery key.
Aug 31 11:40:46 server-3 corosync[1868]:   [WD    ] no resources configured.
Aug 31 11:40:46 server-3 corosync[1868]:   [SERV  ] Service engine loaded: corosync watchdog service [7]

Ich habe den 3. Server schon x mal frisch installiert, langsam weiß ich nicht weiter. Das Quorum scheint das Problem zu sein, aber die anderen zwei Nodes laufen doch und müssten ihr "ok" geben?

Status vom neuen, 3. Node::

Code: 
root@server-3 ~ # pvecm status
Cluster information
-------------------
Name:             MeinCluster
Config Version:   52
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Wed Aug 31 11:47:11 2022
Quorum provider:  corosync_votequorum
Nodes:            1
Node ID:          0x00000002
Ring ID:          2.5
Quorate:          No

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      1
Quorum:           2 Activity blocked
Flags:

Membership information
----------------------
    Nodeid      Votes Name
0x00000002          1 3.3.3.3 (local)
root@server-3 ~ #

Status vom Cluster:

Code: 
root@server-2 ~ # pvecm status
Cluster information
-------------------
Name:             MeinCluster
Config Version:   52
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Wed Aug 31 11:42:10 2022
Quorum provider:  corosync_votequorum
Nodes:            2
Node ID:          0x00000001
Ring ID:          1.53a94
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      2
Quorum:           2
Flags:            Quorate

Membership information
----------------------
    Nodeid      Votes Name
0x00000001          1 2.2.2.2 (local)
0x00000004          1 1.1.1.1
root@server-2 ~ #

Hat jemand eine Idee, was hier faul ist?

Ein Stück weiter: auf dem neuen, 3. Node habe ich "pvecm expected 1" eingegeben, damit ging der "pvecm add .." Befehl oben weiter:

Code: 
waiting for quorum...OK
(re)generate node files
generate new node certificate
merge authorized SSH keys and known hosts
generated new node certificate, restart pveproxy and pvedaemon services
successfully added node 'rigel-3' to cluster.
root@server-3 ~ #

Dann habe ich auf allen Server

systemctl restart corosync.service
systemctl restart pve-cluster.service

gemacht, aber der neue Node ist trotzdem noch nicht grün. An der Ausgabe von pvecm status hat sich nichts verändert.

War noch eine fehlende IP in einer Security-Gruppe.
 
Last edited:
You must log in or register to reply here.
Top Bottom