Clusternode not reachable via GUI

R

rhagemann

New Member
Aug 11, 2023
10
0
1
Hi,

i created my first Proxmox Cluster with a Node with existing VMs and a new Node without VMs.
I cannot reach the NEWNODE via https://newnode:8006 and in my cluster the OLDNODE cannot load data from the NEWNODE and get this message:
Code: 
Connection timed out (596)
At creation of the cluster both could not reach each other via SSH. But now they can.
I restarted pvedaemon on both Nodes.I restarted pvedaemon on both Nodes but nothing changed.

Now I see that i cannot get "Cluster Join Information". This button is grey. So i googled
Code: 
pvesh get /cluster/config/join
unable to read certificate from '/etc/pve/nodes/NEWNODE/pve-ssl.pem'

Code: 
root@NEWNODE:~# cat /etc/pve/nodes/NEWNODE/pve-ssl.pem
NULL
so i wanted to create new Certs
Code: 
root@NEWNODE:~# pvecm updatecerts --force
(re)generate node files
generate new node certificate
Certificate request self-signature ok
subject=OU = PVE Cluster Node, O = Proxmox Virtual Environment, CN = NEWNODE.dimedis.de
CA certificate and CA private key do not match
00752F2D8E7F0000:error:05800074:x509 certificate routines:X509_check_private_key:key values mismatch:../crypto
/x509/x509_cmp.c:405:
unable to generate pve ssl certificate:
command 'faketime yesterday openssl x509 -req -in /tmp/pvecertreq-8741.tmp -days 730 -out /etc/pve/nodes/NEWNO
DE/pve-ssl.pem -CAkey /etc/pve/priv/pve-root-ca.key -CA /etc/pve/pve-root-ca.pem -CAserial /etc/pve/priv/pve-r
oot-ca.srl -extfile /tmp/pvesslconf-8741.tmp' failed: exit code 1
And now i dont know how can I fix this.



Code: 
root@NEWNODE:~# systemctl status pve-cluster.service

● pve-cluster.service - The Proxmox VE cluster filesystem
Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; preset: enabled)
Active: active (running) since Fri 2023-08-11 14:22:49 CEST; 1h 48min ago
    Process: 1373 ExecStart=/usr/bin/pmxcfs (code=exited, status=0/SUCCESS)
   Main PID: 1384 (pmxcfs)
      Tasks: 8 (limit: 309282)
     Memory: 42.0M
        CPU: 10.293s
     CGroup: /system.slice/pve-cluster.service
└─1384 /usr/bin/pmxcfs

Aug 11 14:24:11 NEWNODE pmxcfs[1384]: [status] notice: received log
Aug 11 14:39:11 NEWNODE pmxcfs[1384]: [status] notice: received log
Aug 11 14:42:34 NEWNODE pmxcfs[1384]: [dcdb] notice: data verification successful
Aug 11 14:54:12 NEWNODE pmxcfs[1384]: [status] notice: received log
Aug 11 15:09:12 NEWNODE pmxcfs[1384]: [status] notice: received log
Aug 11 15:24:13 NEWNODE pmxcfs[1384]: [status] notice: received log
Aug 11 15:39:14 NEWNODE pmxcfs[1384]: [status] notice: received log
Aug 11 15:42:34 NEWNODE pmxcfs[1384]: [dcdb] notice: data verification successful
Aug 11 15:54:15 NEWNODE pmxcfs[1384]: [status] notice: received log
Aug 11 16:09:15 NEWNODE pmxcfs[1384]: [status] notice: received log


Code: 
root@NEWNODE:~# systemctl status corosync

● corosync.service - Corosync Cluster Engine
Loaded: loaded (/lib/systemd/system/corosync.service; enabled; preset: enabled)
Active: active (running) since Fri 2023-08-11 14:22:49 CEST; 1h 48min ago
       Docs: man:corosync
             man:corosync.conf
             man:corosync_overview
   Main PID: 1460 (corosync)
      Tasks: 9 (limit: 309282)
     Memory: 133.6M
        CPU: 2min 27.105s
     CGroup: /system.slice/corosync.service
└─1460 /usr/sbin/corosync -f

Aug 11 14:22:52 NEWNODE corosync[1460]:   [QUORUM] This node is within the primary component and will provide service.
Aug 11 14:22:52 NEWNODE corosync[1460]:   [QUORUM] Members[2]: 1 2
Aug 11 14:22:52 NEWNODE corosync[1460]:   [MAIN  ] Completed service synchronization, ready to provide service.
Aug 11 14:22:52 NEWNODE corosync[1460]:   [KNET  ] pmtud: PMTUD link change for host: 1 link: 0 from 469 to 1397
Aug 11 14:22:52 NEWNODE corosync[1460]:   [KNET  ] pmtud: Global data MTU changed to: 1397
Aug 11 14:22:55 NEWNODE corosync[1460]:   [KNET  ] rx: host: 1 link: 1 is up
Aug 11 14:22:55 NEWNODE corosync[1460]:   [KNET  ] link: Resetting MTU for link 1 because host 1 joined
Aug 11 14:22:55 NEWNODE corosync[1460]:   [KNET  ] host: host: 1 (passive) best link: 0 (pri: 1)
Aug 11 14:22:55 NEWNODE corosync[1460]:   [KNET  ] pmtud: PMTUD link change for host: 1 link: 1 from 469 to 1397
Aug 11 14:22:55 NEWNODE corosync[1460]:   [KNET  ] pmtud: Global data MTU changed to: 1397
Code: 
● pveproxy.service - PVE API Proxy Server
     Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; preset: enabled)
     Active: active (running) since Fri 2023-08-11 14:22:56 CEST; 1h 51min ago
    Process: 1499 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=0/SUCCESS)
    Process: 1504 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
   Main PID: 1507 (pveproxy)
      Tasks: 4 (limit: 309282)
     Memory: 140.8M
        CPU: 3min 47.063s
     CGroup: /system.slice/pveproxy.service
             ├─ 1507 pveproxy
├─17512 "pveproxy worker"
├─17513 "pveproxy worker"
└─17514 "pveproxy worker"

Aug 11 16:14:48 NEWNODE pveproxy[1507]: starting 1 worker(s)
Aug 11 16:14:48 NEWNODE pveproxy[1507]: worker 17512 started
Aug 11 16:14:48 NEWNODE pveproxy[1507]: worker 17511 finished
Aug 11 16:14:48 NEWNODE pveproxy[1507]: worker 17510 finished
Aug 11 16:14:48 NEWNODE pveproxy[1507]: starting 2 worker(s)
Aug 11 16:14:48 NEWNODE pveproxy[1507]: worker 17513 started
Aug 11 16:14:48 NEWNODE pveproxy[1507]: worker 17514 started
Aug 11 16:14:48 NEWNODE pveproxy[17512]: /etc/pve/local/pve-ssl.pem: failed to use local certificate chain (cert_file o
r cert) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 2009.
Aug 11 16:14:48 NEWNODE pveproxy[17513]: /etc/pve/local/pve-ssl.pem: failed to use local certificate chain (cert_file o
r cert) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 2009.
Aug 11 16:14:48 NEWNODE pveproxy[17514]: /etc/pve/local/pve-ssl.pem: failed to use local certificate chain (cert_file o
r cert) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 2009.



Can you help me to connect to my new Server?
Do you need more information?
 
Last edited:
Hi,

Did you see an error when joining the new node to that cluster?
Are they both nodes with the same Proxmox version?

Can you post the output of the following commands:
Bash: 
pvecm status

cat /etc/pve/corosync.conf

# Regarding the certificates:
mount |grep '/etc/pve'
stat /etc/pve/nodes/pve5-9/pve-ssl.key
pvesh get /cluster/config/join
 
Did you see an error when joining the new node to that cluster?
Click to expand...

In /var/log/pve/tasks/index I see this error:
Code: 
UPID:NEWNODE:00341E37:073487DE:64D6030B:clusterjoin::root@pam: 64D6031D unable to generate pve ssl certificate: command 'faketime yesterday openssl x509 -req -in /tmp/pvecertreq-3415607.tmp -days 730 -out /etc/pve/nodes/NEWNODE/pve-ssl.pem -CAkey /etc/pve/priv/pve-root-ca.key -CA /etc/pve/pve-root-ca.pem -CAserial /etc/pve/priv/pve-root-ca.srl -extfile /tmp/pvesslconf-3415607.tmp' failed: exit code 1

Are they both nodes with the same Proxmox version?
Click to expand...
I see my first Node has a very old version: 6.3-3 and my new one has 8.0.3.


Code: 
root@NEWNODE:~# pvecm status
Cluster information
-------------------
Name:             cluster-dmz01
Config Version:   2
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Mon Aug 14 12:08:23 2023
Quorum provider:  corosync_votequorum
Nodes:            2
Node ID:          0x00000002
Ring ID:          1.28
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   2
Highest expected: 2
Total votes:      2
Quorum:           2
Flags:            Quorate

Membership information
----------------------
    Nodeid      Votes Name
0x00000001          1 192.168.212.65
0x00000002          1 192.168.212.79 (local)



Code: 
root@NEWNODE:~# cat /etc/pve/corosync.conf
logging {
  debug: off
  to_syslog: yes
}

nodelist {
  node {
    name: NEWNODE
    nodeid: 2
    quorum_votes: 1
    ring0_addr: 192.168.212.79
    ring1_addr: 194.8.212.79
  }
  node {
    name: OLDNODE
    nodeid: 1
    quorum_votes: 1
    ring0_addr: 192.168.212.65
    ring1_addr: 194.8.212.65
  }
}

quorum {
  provider: corosync_votequorum
}

totem {
  cluster_name: cluster-dmz01
  config_version: 2
  interface {
    linknumber: 0
  }
  interface {
    linknumber: 1
  }
  ip_version: ipv4-6
  link_mode: passive
  secauth: on
  version: 2
}

Code: 
root@NEWNODE:~# mount |grep '/etc/pve'
/dev/fuse on /etc/pve type fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other)

Code: 
root@NEWNODE:~# stat /etc/pve/nodes/pve5-9/pve-ssl.key
stat: cannot statx '/etc/pve/nodes/pve5-9/pve-ssl.key': No such file or directory
Code: 
root@NEWNODE:~# pvesh get /cluster/config/join
unable to read certificate from '/etc/pve/nodes/NEWNODE/pve-ssl.pem'
 
Hello,

Thank you for the outputs!


rhagemann said:
I see my first Node has a very old version: 6.3-3 and my new one has 8.0.3.
Click to expand...
Make sure that you have a backup of all important VMs and files!

In general, the nodes should have the same version. In this case, I would try to do the following commands:

Bash: 
cd /etc/pve/local

mv pve-ssl.key pve-ssl.key-old

mv pve-ssl.pem pve-ssl.pem-old 

pvecm updatecerts --force

service pveproxy restart`
 
Code: 
(re)generate node files
generate new node certificate
Certificate request self-signature ok
subject=OU = PVE Cluster Node, O = Proxmox Virtual Environment, CN = NEWNODE.dimedis.de
CA certificate and CA private key do not match
00D534A8A47F0000:error:05800074:x509 certificate routines:X509_check_private_key:key values mismatch:../crypto/x509/x509_cmp.c:405:
unable to generate pve ssl certificate:
command 'faketime yesterday openssl x509 -req -in /tmp/pvecertreq-775708.tmp -days 730 -out /etc/pve/nodes/NEWNODE/pve-ssl.pem -CAkey /etc/pve/priv/pve-root-ca.key -CA /etc/pve/pve-root-ca.pem -CAserial /etc/pve/priv/pve-root-ca.srl -extfile /tmp/pvesslconf-775708.tmp' failed: exit code 1

I think i will remove the Node from Cluster and reinstall it.
Thank you for your help.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom