Adding 6.2 nodes to 6.0 cluster - Error 500 Can't connect to x.x.x.x:8006 (hostname verification failed)

Dan S

Member
Nov 12, 2020
5
0
6
46
Hi.

I'm adding new nodes to a 6.0 Proxmox + CEPH cluster.
The new nodes are with the version 6.2
The idea is to add the new 6.2 nodes to the actual 6.0 cluster, and the upgrade the 6.0 nodes to 6.2. Finally jump to version 6.3

But when i try to add a new node (v6.2) to the 6.0 cluster i get a verification error (i suppose is related to the certificate):

Bash:
root@newnode3:~# pvecm add x.x.x.4
Please enter superuser (root) password for 'x.x.x.4': ********
Establishing API connection with host 'x.x.x.4'
The authenticity of host 'x.x.x.4' can't be established.
X509 SHA256 key fingerprint is XX:XX:XX:XX:XX:XX:XX:XX..........................
Are you sure you want to continue connecting (yes/no)? yes
500 Can't connect to x.x.x.4:8006 (hostname verification failed)

The /etc/hosts has the correct name/ip and it's the same all the hosts (running ones and the ones to be added).

Shoudl i add it with the --use_ssh option to skip the verification?

pvecm add x.x.x.4 --use_ssh 1

What should i do to skip the problem and add the new nodes to the cluster?

Thanks in advance.

Daniel.
 
Last edited:

Petr Svacina

Active Member
Oct 1, 2018
31
10
28
46
Hi, I have thew same problem. Two nodes in cluster last enterprise update and third can not be added via cli. Adding via GUI breaks cluster ...
Any idea ? Is this SAFE ?

Code:
pvecm add x.x.x.4 --use_ssh 1
 

Petr Svacina

Active Member
Oct 1, 2018
31
10
28
46
I have also tried to run on working cluster and even on the new node, not helped ....:

Code:
pvecm updatecerts
 

Petr Svacina

Active Member
Oct 1, 2018
31
10
28
46
UPDATE:

I have just solved the problem with adding all servers in cluster to /etc/hosts on the new node and after I was able to add new node to cluster by hostname:

Code:
pvecm add hostname-no-ip-of-some-working-node
 

Dan S

Member
Nov 12, 2020
5
0
6
46
UPDATE:

I have just solved the problem with adding all servers in cluster to /etc/hosts on the new node and after I was able to add new node to cluster by hostname:

Code:
pvecm add hostname-no-ip-of-some-working-node

I already have the /etc/host in all the servers reflecting the names and IP.
So you added with the name of one server already in the cluster, as reflected in /etc/hosts instread of the IP that is the usual, no?

So if the exisisting server in /etc/hosts

1.1.1.10 server10 1.1.1.11 newserver11

you used from the new server (1.1.1.11)

Code:
pvecm add server10

instead of

Code:
pvecm add 1.1.1.10

And that solved the problem. Is it ok?

Thanks a lot.

Daniel.
 

Dan S

Member
Nov 12, 2020
5
0
6
46
I tried that and it didn't worked for me. Is there something else I can do?
It worked with no problems.

Have you added the hostnames en every /etc/hosts on all the nodes, and added the new node from command line with "pvecm add ip.actual.node" from the new node?

Have you checked that can log with SSH from the new node to the actual ones?
 

Petr Svacina

Active Member
Oct 1, 2018
31
10
28
46
This bug does not exist anymore, if your system is up-to-date ... So my questions are: Do you have:

1. /etc/hosts - correct values on all hosts ?
2. all hosts are up-to-date ?
 
Last edited:

danielo515

Member
Nov 20, 2020
28
1
8
36
I don't have all hosts on /etc/hosts of all hosts. I just added them to the node I want to join.
The cluster is on 6.2-12 while the new node is on 6.2-4
Previously I tried the new node on 6.3-1 but that didn't worked, so I just downgraded to the latest 6.2 is I found.

Indeed the ssh may be the problem:


Code:
root@proxmox-virtual:~# ssh -vv virtual
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1d  10 Sep 2019
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving "virtual" port 22
debug2: ssh_connect_direct
debug1: Connecting to virtual [192.168.0.6] port 22.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type 0
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.9p1 Debian-10+deb10u2


that is from the new node I want to join to the cluster original node
 

alterman1994

New Member
Aug 6, 2021
15
2
3
28
Guys, has anybody managed to address this? I have exactly the same issue, trying to add 2 more nodes with the same version (6.4-13) and getting:
root@pve6:~# pvecm add pve1
Please enter superuser (root) password for 'pve1': *******************
Establishing API connection with host 'pve1'
500 Can't connect to pve1:8006 (hostname verification failed)

I used the "pvecm add pve1" previously and it was working fine (current cluster contains 4 nodes)


/etc/hosts contains all correct records on ALL nodes (old and new ones). I tested with SSH from all nodes to all nodes - all good, Network is stable.
 

alterman1994

New Member
Aug 6, 2021
15
2
3
28
guys, I've figured this out - it's because of Let's Encrypt SSL certificate installed on the cluster node thus you need to use covered server's hostname (in my case it's pve1.mydomain.com):

Bash:
root@pve6:~# pvecm add pve1.mydomain.com
Please enter superuser (root) password for 'pve1.mydomain.com': *******************
Establishing API connection with host 'pve1.mydomain.com'
Login succeeded.
check cluster join API version
No cluster network links passed explicitly, fallback to local node IP '192.168.200.246'
Request addition of this node
Join request OK, finishing setup locally
stopping pve-cluster service
backup old database to '/var/lib/pve-cluster/backup/config-1632474152.sql.gz'
waiting for quorum...OK
(re)generate node files
generate new node certificate
merge authorized SSH keys and known hosts
generated new node certificate, restart pveproxy and pvedaemon services
successfully added node 'pve6' to cluster.

hope someone will find it useful ;)
 
  • Like
Reactions: IMW72

IMW72

New Member
Sep 6, 2020
6
1
3
43
pvecm add node1.domain.com -force

Using the ip of node, I got errors. (Proxmox 6.4-13)

worked for me with hostname !
Thanks
 
  • Like
Reactions: alterman1994

de Thysebaert

Active Member
Mar 12, 2017
42
3
28
65
Hi,
I had also the same issue.
All nodes in /etc/hosts with correct IP (for me public IP with lets'encrypt certificates and private IP for the quorum in a dedicated vlan)
using short name to join the cluster failed
using FQDN resolve the issue and my new node si now part of the cluster.
Thx for this post
 

jsterr

Active Member
Jul 24, 2020
243
45
33
31
This is still existent on pve 7.2 when adding a node via cli and IP.

Please enter superuser (root) password for '10.2.121.212': ********
Establishing API connection with host '10.2.121.212'
The authenticity of host '10.2.121.212' can't be established.
X509 SHA256 key fingerprint is 20:A0:2F:64:AC:03:6B:1B:1F:4D:5D:55:E4:9B:7B:B1:7F:A9:3B:27:F4:1A:C6:35:B2:1C:58:DE:A8:E6:A6:F9.
Are you sure you want to continue connecting (yes/no)?
TASK ERROR: 500 Can't connect to 10.2.121.212:8006 (hostname verification failed)

pvecm add hostnameHostname works without problems.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!