[SOLVED] Versehentlich Cluster hinzugefügt - nichts geht mehr (Cannot initialize CMAP service)

[Dark18]

Hallo zusammen,

ich bin recht neu was Proxmox angeht und habe auf meinem Server gestern aus Neugier ein Cluster hinzugefügt.
Nur.. scheint es mir einiges kaputt gemacht zu haben. Nichtmal vorhandene VM's kann ich noch starten.

Im Internet habe ich bereits recherchiert, aber keine Lösung gefunden.

Hier einige Informationen:

Spoiler: pvecm nodes

Cannot initialize CMAP service

Spoiler: systemctl restart corosync

Job for corosync[PUNKT]service failed because the control process exited with error code.
See "systemctl status corosync.service" and "journalctl -xe" for details.

Spoiler: journalctl -b -u corosync

-- Logs begin at Wed 2019-05-08 17:54:52 CEST, end at Mon 2019-05-13 19:01:29 CEST. --
May 13 17:53:55 privserv systemd[1]: Starting Corosync Cluster Engine...
May 13 17:53:55 privserv corosync[1959]: [MAIN ] Corosync Cluster Engine ('2.4.4-dirty'): started and ready to provide service.
May 13 17:53:55 privserv corosync[1959]: [MAIN ] Corosync built-in features: dbus rdma monitoring watchdog systemd xmlconf qdevices qnetd snmp pie relro bindnow
May 13 17:53:55 privserv corosync[1959]: notice [MAIN ] Corosync Cluster Engine ('2.4.4-dirty'): started and ready to provide service.
May 13 17:53:55 privserv corosync[1959]: info [MAIN ] Corosync built-in features: dbus rdma monitoring watchdog systemd xmlconf qdevices qnetd snmp pie relro bindnow
May 13 17:53:55 privserv corosync[1959]: warning [MAIN ] interface section bindnetaddr is used together with nodelist. Nodelist one is going to be used.
May 13 17:53:55 privserv corosync[1959]: warning [MAIN ] Please migrate config file to nodelist.
May 13 17:53:55 privserv corosync[1959]: [MAIN ] interface section bindnetaddr is used together with nodelist. Nodelist one is going to be used.
May 13 17:53:55 privserv corosync[1959]: [MAIN ] Please migrate config file to nodelist.
May 13 17:53:55 privserv corosync[1959]: notice [TOTEM ] Initializing transport (UDP/IP Multicast).
May 13 17:53:55 privserv corosync[1959]: notice [TOTEM ] Initializing transmit/receive security (NSS) crypto: aes256 hash: sha1
May 13 17:53:55 privserv corosync[1959]: [TOTEM ] Initializing transport (UDP/IP Multicast).
May 13 17:53:55 privserv corosync[1959]: [TOTEM ] Initializing transmit/receive security (NSS) crypto: aes256 hash: sha1
May 13 17:53:55 privserv corosync[1959]: notice [TOTEM ] The network interface is down.
May 13 17:53:55 privserv corosync[1959]: [TOTEM ] The network interface is down.
May 13 17:53:55 privserv corosync[1959]: notice [SERV ] Service engine loaded: corosync configuration map access [0]
May 13 17:53:55 privserv corosync[1959]: info [QB ] server name: cmap
May 13 17:53:55 privserv corosync[1959]: notice [SERV ] Service engine loaded: corosync configuration service [1]
May 13 17:53:55 privserv corosync[1959]: info [QB ] server name: cfg
May 13 17:53:55 privserv corosync[1959]: notice [SERV ] Service engine loaded: corosync cluster closed process group service v1.01 [2]
May 13 17:53:55 privserv corosync[1959]: info [QB ] server name: cpg
May 13 17:53:55 privserv corosync[1959]: notice [SERV ] Service engine loaded: corosync profile loading service [4]
May 13 17:53:55 privserv corosync[1959]: notice [SERV ] Service engine loaded: corosync resource monitoring service [6]
May 13 17:53:55 privserv corosync[1959]: warning [WD ] Watchdog not enabled by configuration
May 13 17:53:55 privserv corosync[1959]: warning [WD ] resource load_15min missing a recovery key.
May 13 17:53:55 privserv corosync[1959]: warning [WD ] resource memory_used missing a recovery key.
May 13 17:53:55 privserv corosync[1959]: info [WD ] no resources configured.
May 13 17:53:55 privserv corosync[1959]: notice [SERV ] Service engine loaded: corosync watchdog service [7]
May 13 17:53:55 privserv corosync[1959]: notice [QUORUM] Using quorum provider corosync_votequorum
May 13 17:53:55 privserv corosync[1959]: crit [QUORUM] Quorum provider: corosync_votequorum failed to initialize.
May 13 17:53:55 privserv corosync[1959]: error [SERV ] Service engine 'corosync_quorum' failed to load for reason 'configuration error: nodelist or quorum.expected_votes must be configured!'
May 13 17:53:55 privserv corosync[1959]: error [MAIN ] Corosync Cluster Engine exiting with status 20 at service.c:356.
May 13 17:53:55 privserv systemd[1]: corosync.service: Main process exited, code=exited, status=20/n/a
May 13 17:53:55 privserv systemd[1]: Failed to start Corosync Cluster Engine.
May 13 17:53:55 privserv systemd[1]: corosync.service: Unit entered failed state.
May 13 17:53:55 privserv systemd[1]: corosync.service: Failed with result 'exit-code'.
May 13 19:00:56 privserv systemd[1]: Starting Corosync Cluster Engine...
May 13 19:00:56 privserv corosync[6384]: [MAIN ] Corosync Cluster Engine ('2.4.4-dirty'): started and ready to provide service.
May 13 19:00:56 privserv corosync[6384]: [MAIN ] Corosync built-in features: dbus rdma monitoring watchdog systemd xmlconf qdevices qnetd snmp pie relro bindnow
May 13 19:00:56 privserv corosync[6384]: notice [MAIN ] Corosync Cluster Engine ('2.4.4-dirty'): started and ready to provide service.
May 13 19:00:56 privserv corosync[6384]: info [MAIN ] Corosync built-in features: dbus rdma monitoring watchdog systemd xmlconf qdevices qnetd snmp pie relro bindnow
May 13 19:00:56 privserv corosync[6384]: warning [MAIN ] interface section bindnetaddr is used together with nodelist. Nodelist one is going to be used.
May 13 19:00:56 privserv corosync[6384]: warning [MAIN ] Please migrate config file to nodelist.
May 13 19:00:56 privserv corosync[6384]: [MAIN ] interface section bindnetaddr is used together with nodelist. Nodelist one is going to be used.
May 13 19:00:56 privserv corosync[6384]: [MAIN ] Please migrate config file to nodelist.
May 13 19:00:56 privserv corosync[6384]: notice [TOTEM ] Initializing transport (UDP/IP Multicast).
May 13 19:00:56 privserv corosync[6384]: notice [TOTEM ] Initializing transmit/receive security (NSS) crypto: aes256 hash: sha1
May 13 19:00:56 privserv corosync[6384]: [TOTEM ] Initializing transport (UDP/IP Multicast).
May 13 19:00:56 privserv corosync[6384]: [TOTEM ] Initializing transmit/receive security (NSS) crypto: aes256 hash: sha1
May 13 19:00:56 privserv corosync[6384]: notice [TOTEM ] The network interface is down.
May 13 19:00:56 privserv corosync[6384]: notice [SERV ] Service engine loaded: corosync configuration map access [0]
May 13 19:00:56 privserv corosync[6384]: info [QB ] server name: cmap
May 13 19:00:56 privserv corosync[6384]: notice [SERV ] Service engine loaded: corosync configuration service [1]
May 13 19:00:56 privserv corosync[6384]: info [QB ] server name: cfg
May 13 19:00:56 privserv corosync[6384]: notice [SERV ] Service engine loaded: corosync cluster closed process group service v1.01 [2]
May 13 19:00:56 privserv corosync[6384]: info [QB ] server name: cpg
May 13 19:00:56 privserv corosync[6384]: notice [SERV ] Service engine loaded: corosync profile loading service [4]
May 13 19:00:56 privserv corosync[6384]: [TOTEM ] The network interface is down.
May 13 19:00:56 privserv corosync[6384]: notice [SERV ] Service engine loaded: corosync resource monitoring service [6]
May 13 19:00:56 privserv corosync[6384]: warning [WD ] Watchdog not enabled by configuration
May 13 19:00:56 privserv corosync[6384]: warning [WD ] resource load_15min missing a recovery key.
May 13 19:00:56 privserv corosync[6384]: warning [WD ] resource memory_used missing a recovery key.
May 13 19:00:56 privserv corosync[6384]: info [WD ] no resources configured.
May 13 19:00:56 privserv corosync[6384]: notice [SERV ] Service engine loaded: corosync watchdog service [7]
May 13 19:00:56 privserv corosync[6384]: notice [QUORUM] Using quorum provider corosync_votequorum
May 13 19:00:56 privserv corosync[6384]: crit [QUORUM] Quorum provider: corosync_votequorum failed to initialize.
May 13 19:00:56 privserv corosync[6384]: error [SERV ] Service engine 'corosync_quorum' failed to load for reason 'configuration error: nodelist or quorum.expected_votes must be configured!'
May 13 19:00:56 privserv corosync[6384]: error [MAIN ] Corosync Cluster Engine exiting with status 20 at service.c:356.
May 13 19:00:56 privserv systemd[1]: corosync.service: Main process exited, code=exited, status=20/n/a
May 13 19:00:56 privserv systemd[1]: Failed to start Corosync Cluster Engine.
May 13 19:00:56 privserv systemd[1]: corosync.service: Unit entered failed state.
May 13 19:00:56 privserv systemd[1]: corosync.service: Failed with result 'exit-code'.

Spoiler: cat /etc/hosts

127.0.0.1 localhost.localdomain localhost
176.*.**.*4 privserv

::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
2a01:***:***:***::2 privserv

Spoiler: cat /etc/pve/corosync.conf

logging {
debug: off
to_syslog: yes
}

nodelist {
node {
name: privserv
nodeid: 1
quorum_votes: 1
ring0_addr: 12
}
}

quorum {
provider: corosync_votequorum
}

totem {
cluster_name: 12
config_version: 1
interface {
bindnetaddr: 12
ringnumber: 0
}
ip_version: ipv4
secauth: on
version: 2
}

Spoiler: ls -lh /etc/pve/

total 2.5K
-r--r----- 1 root www-data 451 May 8 18:31 authkey.pub
-r--r----- 1 root www-data 338 May 12 17:36 corosync.conf
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 local -> nodes/privserv
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 lxc -> nodes/privserv/lxc
dr-xr-xr-x 2 root www-data 0 May 8 18:31 nodes
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 openvz -> nodes/privserv/openvz
dr-x------ 2 root www-data 0 May 8 18:31 priv
-r--r----- 1 root www-data 2.1K May 8 18:31 pve-root-ca.pem
-r--r----- 1 root www-data 1.7K May 8 18:31 pve-www[PUNKT]key
lr-xr-xr-x 1 root www-data 0 Jan 1 1970 qemu-server -> nodes/privserv/qemu-server
-r--r----- 1 root www-data 119 May 8 18:31 vzdump [PUNKT] cron

Was kann ich tun? Wie kann ich es wieder "rückgängig" machen?

Danke schonmal für jede Hilfe

LG

 

[hitman]

Ähm, lösch doch einfach den cluster, ein Cluster mit einer Node hat kein Stimmrecht, daher startet auch keine VM mehr.

https://pve.proxmox.com/wiki/Cluster_Manager#_remove_a_cluster_node

 

[Dark18]

Das Problem ist, dass ich diese Befehle nicht nutzen kann:

root@privserv ~ # pvecm nodes
Cannot initialize CMAP service

 

[Dark18]

So, kleines Update: Ich habe jetzt die unteren Befehle genutzt (Separate A Node Without Reinstalling), bekomme aber wieder Fehler:

root@privserv ~ # systemctl stop pve-cluster
root@privserv ~ # systemctl stop corosync
root@privserv ~ # pmxcfs -l
[main] notice: forcing local mode (although corosync.conf exists)
root@privserv ~ # rm /etc/pve/corosync.conf
root@privserv ~ # rm /etc/corosync/*
rm: cannot remove '/etc/corosync/uidgid.d': Is a directory
root@privserv ~ # killall pmxcfs
root@privserv ~ # systemctl start pve-cluster
root@privserv ~ # pvecm delnode oldnode
Node/IP: oldnode is not a known host of the cluster.
root@privserv ~ # pvecm delnode privserv
Cannot delete myself from cluster!
root@privserv ~ # pvecm expected 1
Corosync config '/etc/pve/corosync.conf' does not exist - is this node part of a cluster?
Cannot initialize CMAP service
root@privserv ~ #

 

[Dark18]

Nach einem Reboot scheint jetzt alles zu funktionieren. Danke!