Cluster in "Standalone node - no cluster defined"

liemba · Feb 14, 2019

Hi,

I have a Cluster that was fine and now is "Standalone node - no cluster defined" in the GUI. It seems to operate normal. I have some issues that might be related or not (HA seems to be affected). I have upgraded to the latest "5.3-9" and still have the no cluster defined issue.

"/etc/corosync/corosync.conf" are identical on all systems - and has names and ips correct.

"pvecm status" outputs

Code:

Quorum information
------------------
Date:             Thu Feb 14 08:31:55 2019
Quorum provider:  corosync_votequorum
Nodes:            3
Node ID:          0x00000001
Ring ID:          1/256
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   5
Highest expected: 5
Total votes:      3
Quorum:           3 
Flags:            Quorate

Membership information
----------------------
    Nodeid      Votes Name
0x00000001          1 10.10.10.1 (local)
0x00000002          1 10.10.10.2
0x00000003          1 10.10.10.5

(2/5 systems are down for repairs)

dcsapak · Feb 14, 2019

what does 'pvesh get /cluster/config/join --output-format json-pretty' show?

liemba · Feb 14, 2019

Hi,

It says "hostname lookup 'blade04' failed - failed to get address info for: blade04: Name or service not known"

And Blad04 is down for maintenance. With blade04 on - I still have the same "no cluster" issue.

liemba · Feb 14, 2019

A screenshot

liemba · Feb 14, 2019

FIXED. Started the two nodes down for maintenance - upgrade them to 5.3-9 - and then the cluster was fine.

Downed another node without the "Standalone node - no cluster defined" issue earlier.

zdenekjanis · Apr 6, 2019

Hello,
I have the same problem.
Cluster 4 node: 3 node online and 1 node offline.
Virtual Environment 5.3-12 (latest available version).
See screenshots:

Help?
Thank you.

dcsapak · Apr 8, 2019

dcsapak said:
what does 'pvesh get /cluster/config/join --output-format json-pretty' show?

@zdenekjanis

zdenekjanis · Apr 8, 2019

dcsapak said:
@zdenekjanis

root@bPVE:~# pvesh get /cluster/config/join --output-format json-pretty
hostname lookup 'iPVE' failed - failed to get address info for: iPVE: Name or service not known

This is expected because iPVE is offline.
It should not be a problem. Or yes?

Thank you.

dcsapak · Apr 8, 2019

zdenekjanis said:
This is expected because iPVE is offline.

this has nothing to do with it being offline, this is an error that host cannot resolve the hostname, which
should always work, regardless if the node is online or not

zdenekjanis · Apr 8, 2019

dcsapak said:
this has nothing to do with it being offline, this is an error that host cannot resolve the hostname, which
should always work, regardless if the node is online or not

I thought it was enough to record in corosync.conf:

nodelist {
node {
name: bPVE
nodeid: 1
quorum_votes: 1
ring0_addr: 2a04:4880:0:4::7083
}
node {
name: gPVE
nodeid: 2
quorum_votes: 1
ring0_addr: 2a04:4880:0:4::7088
}
node {
name: iPVE
nodeid: 3
quorum_votes: 1
ring0_addr: 2a04:4880:0:4::708a
}
node {
name: jPVE
nodeid: 4
quorum_votes: 1
ring0_addr: 2a04:4880:0:4::708b
}
}

Or do you need entries in /etc/hosts?

dcsapak · Apr 8, 2019

for normal inter cluster communications, the node name needs to be resolvable

zdenekjanis · Apr 8, 2019

dcsapak said:
for normal inter cluster communications, the node name needs to be resolvable

All you had to do was add all the nodes to /etc/hosts and see them in the GUI cluster.

Thank you for your help.

okieunix1957 · Feb 21, 2020

Hi, I have similar issue this what the follow command shows me:
host07# pvesh get /cluster/config/join --output-format json-pretty
unknown node 'host07'
host07:/etc/corosync#

host07 is .18 node which is the master node.

host07:~# pvecm nodes

Membership information
----------------------
Nodeid Votes Name
1 1 x.x.x.18 (local)
2 1 x.x.x.19
3 1 x.x.x.20
5 1 x.x.x.22
6 1 x.x.x.23
host07:~#

-host07:~# pvecm status
Quorum information
------------------
Date: Fri Feb 21 03:21:57 2020
Quorum provider: corosync_votequorum
Nodes: 5
Node ID: 0x00000001
Ring ID: 1/3804
Quorate: Yes

Votequorum information
----------------------
Expected votes: 5
Highest expected: 5
Total votes: 5
Quorum: 3
Flags: Quorate

Membership information
----------------------
Nodeid Votes Name
0x00000001 1 x.x.x.18 (local)
0x00000002 1 x.x.x.19
0x00000003 1 x.x.x.20
0x00000005 1 x.x.x..22
0x00000006 1 x.x.x.23
host07:~#

I have restarted both corosync and pve-cluster
any ideas?

dcsapak · Feb 21, 2020

how does your /etc/pve/corosync.conf look like?

okieunix1957 · Feb 21, 2020

dcsapak said:
how does your /etc/pve/corosync.conf look like?

host07:/etc/corosync# more corosync.conf
logging {
debug: off
to_syslog: yes
}

nodelist {
node {
name: host07
nodeid: 1
quorum_votes: 1
ring0_addr: x.x.x18
}
node {
name: host08
nodeid: 2
quorum_votes: 1
ring0_addr: x.x.x.19
}
node {
name: host09
nodeid: 3
quorum_votes: 1
ring0_addr: x.x.x.20
}
node {
name: host11
nodeid: 5
quorum_votes: 1
ring0_addr: x.x.x.22
}
node {
name: x.x.x.host12
nodeid: 6
quorum_votes: 1
ring0_addr: x.x.x.23
}
}

quorum {
provider: corosync_votequorum
}

totem {
cluster_name: sc4-devops-clu2
config_version: 7
interface {
bindnetaddr: x.x.x.18
ringnumber: 0
}
ip_version: ipv4
secauth: on
version: 2
}

host07:/etc/corosync#

okieunix1957 · Feb 21, 2020

I notice that my cluster is named differently than what in my corosync. Where can I change this at?

host07:/etc/pve# cat .members
{
"nodename": "host07",
"version": 7,
"cluster": { "name": "sc4-devops-clu2", "version": 7, "nodes": 5, "quorate": 1 },
"nodelist": {
"host07": { "id": 1, "online": 1, "ip": "x.x.x.18"},
"host08": { "id": 2, "online": 1, "ip": "x.x.x.19"},
"host09": { "id": 3, "online": 1, "ip": "x.x.x.20"},
"host11": { "id": 5, "online": 1, "ip": "x.x.x.22"},
"host12": { "id": 6, "online": 1, "ip": "x.x.x.23"}
}
}
host07:/etc/pve#

I must have changed that somehow or somewhere. I think this is the cause of the problem.
Suggestion?

okieunix1957 · Feb 22, 2020

okieunix1957 said:
host07:/etc/corosync# more corosync.conf
logging {
debug: off
to_syslog: yes
}

nodelist {
node {
name: host07
nodeid: 1
quorum_votes: 1
ring0_addr: x.x.x18
}
node {
name: host08
nodeid: 2
quorum_votes: 1
ring0_addr: x.x.x.19
}
node {
name: host09
nodeid: 3
quorum_votes: 1
ring0_addr: x.x.x.20
}
node {
name: host11
nodeid: 5
quorum_votes: 1
ring0_addr: x.x.x.22
}
node {
name: x.x.x.host12
nodeid: 6
quorum_votes: 1
ring0_addr: x.x.x.23
}
}

quorum {
provider: corosync_votequorum
}

totem {
cluster_name: sc4-devops-clu2
config_version: 7
interface {
bindnetaddr: x.x.x.18
ringnumber: 0
}
ip_version: ipv4
secauth: on
version: 2
}

host07:/etc/corosync#

I got this resolved. By cleaning up the node 10 I added and clean up the corosync.
Corosync is far too sensitive.

elamigosam · Aug 25, 2020

dcsapak said:
what does 'pvesh get /cluster/config/join --output-format json-pretty' show?

I have the same " Standalone node - no cluster defined " issue after adding a new member (pve03) to the cluster, current member are 6.1 new member is 6.2

I get this after running pvesh get /cluster/config/join --output-format json-pretty:

Any ideas?

Also, other symptoms:
when I connect my third (pve03) node to the network, the web interface of the other nodes gets slowed down, and sometimes the other node desn't even come up online, although it is. After I unplug (physically) the pve03 node from the switch, everything comes back to normal.

Thank you.

Mike84 · Sep 9, 2020

elamigosam said:
I have the same " Standalone node - no cluster defined " issue after adding a new member (pve03) to the cluster, current member are 6.1 new member is 6.2

I get this after running pvesh get /cluster/config/join --output-format json-pretty:
View attachment 19386
Any ideas?

Also, other symptoms:
when I connect my third (pve03) node to the network, the web interface of the other nodes gets slowed down, and sometimes the other node desn't even come up online, although it is. After I unplug (physically) the pve03 node from the switch, everything comes back to normal.

Thank you.

Good morning,

I am experiencing the same issue (local certificate does not exist) but have not found the issue yet.
Can anyone give a hint on this?

Best regards,
Mike

Mike84 · Sep 9, 2020

Good morning,

Short Update:

I found two issues yet.
1) NTP was not in sync on the new host
2) the new node dropped it's network configuration (both KNET rings offline).
Found it out by restarting corosync on the new node and running:

Code:

grep corosync /var/log/syslog on the node

After it tells, all links to all nodes are down (but pingable) I connected to all nodes and ran:

Code:

grep KNET syslog

This way I found out the master node reports both links to the new node are unreachable.

Restarting the network on the new node using

Code:

systemctl restart networking

took it offline,
so I am expecting a configuration issue. I am going to connect to the server using the service controller now
and going to update this thread when I found the bug.

Best regards,
Mike

Cluster in "Standalone node - no cluster defined"

New Member

Proxmox Staff Member

New Member

New Member

Attachments

New Member

Member

Proxmox Staff Member

Member

Proxmox Staff Member

Member

Proxmox Staff Member

Member

Member

Proxmox Staff Member

Member

Member

Member

Member

New Member

New Member