[SOLVED] Messed up cluster

[Kieeps]

So i got a new server a while back, installed proxmox on it and created a cluster and joined my old proxmox server to the cluster, that way i could migrate my VMs and LXC containers very easy... then i figured i'd reinstall the old server, disable the raid controller and have proxmox handle the array. fist i just reinstalled, the same IP and everything... this messed up the cluster badly so i reinstalled again with a new IP, but i cannot remove the old node from the cluster the new server created...

this is the info from the new server:

pvecm nodes

Membership information
----------------------
Nodeid Votes Name
2 1 192.168.1.3 (local)

cat /etc/corosync/corosync.conf
logging {
debug: off
to_syslog: yes
}

nodelist {
node {
name: MegaSurvurr
nodeid: 2
quorum_votes: 1
ring0_addr: 192.168.1.3
}
node {
name: MicroSurvur
nodeid: 1
quorum_votes: 1
ring0_addr: 192.168.1.2
}
}

quorum {
provider: corosync_votequorum
}

totem {
cluster_name: Home
config_version: 4
interface {
bindnetaddr: 192.168.1.2
ringnumber: 0
}
ip_version: ipv4
secauth: on
version: 2
}



ls /etc/pve/nodes/
MegaSurvurr


i tried to do the following:

systemctl stop pve-cluster corosync
*remove MicroServer from the corosync.conf*
systemctl start pve-cluster corosync

and MicroServer is back in the corosync.conf


how do i manualy remove the old node?


There are some other problems that happened to the new server when all this went down.... the VMs seems to be gone :/ the Disk images are still on the disk, but i cant recreate any VMs since the "create VM" menu wont let me pick a node to create it on anymore.

I dont know what kind of info that would be viable in this situation so pleace tell me any commands you need and i'll post it asap.

 

[wolfgang]

Hi,

I guess you lost the quorum and this is the reason why you can't do anything.
With this command, you are able to edit the files

pvesm expected 1

For more information see the man pvesm.

 

[Kieeps]

Thanks for the reply Wolfgang

well, it feals like a permission issue or something...
i suspect you where referring to pvecm not pvesm although setting "expected 1" didn't help much here, i honestly don't know where to start with troubleshooting. first of i'd like to remove the old server from the cluster.... the cluster that the new server forgot for some reason....

basicly the problem is this:

1# the GUI shows the old (MicroSurvurr) as part of the cluster. IMAGE

2# pvecm nodes shows only the local node, so i cant do "pvecm delnode" since there is no more nodes

3# corosync.conf shows both nodes for some reason.

4# When i try to Create a cluster on the new server i get "cluster config '/etc/pve/corosync.conf' already exists (500)" so that wont work.

5# when i try to join a cluster i get:
detected the following error(s):
* authentication key '/etc/corosync/authkey' already exists
* cluster config '/etc/pve/corosync.conf' already exists
* corosync is already running, is this node already in a cluster?!
TASK ERROR: Check if node may join a cluster failed!

Read about people who added new nodes so that they could remove the old ones but i cant seem to do that :/

I had some VMs running before all this, i can see the VM disk images on the array but cannot create VMs on the new server, is it possible to move the disk images to the old server again and recreate a VM with those images as templates?


EDIT: Finally! i followed THIS guide and got the node to disappear, i'm now able to create VMs again the old VMs are gone though but i'll try to recreate the VMs from the images thats still on the drive, and after that i will not touch anything for a while ;D