proxmox CA certificate expired

[term]

I've been running a cluster for more than 10 years now. Yesterday I added a new node and deleted a old one. The new node's gui does not load because it can't generate certificates, and that is because the proxmox ca certificate has expired. I've tried running pvecm updatecerts -f, and have posted the results from the daily update service below.

The old nodes are still accessible via the web gui and work fine.

What is the proper way to regenerate the CA cert? I believe once that is done, I should be able to go to each node and run pvecm updatecerts -f.

Thanks!

root@proxmox6:~# pvecm updatecerts -f
(re)generate node files
generate new node certificate
CA expires in less than 2 weeks, unable to generate certificate.

root@proxmox6:~# systemctl status pve-daily-update.service
● pve-daily-update.service - Daily PVE download activities
     Loaded: loaded (/lib/systemd/system/pve-daily-update.service; static)
     Active: inactive (dead) since Fri 2022-10-28 02:54:37 CDT; 6h ago
TriggeredBy: ● pve-daily-update.timer
    Process: 1526907 ExecStart=/usr/bin/pveupdate (code=exited, status=0/SUCCESS)
   Main PID: 1526907 (code=exited, status=0/SUCCESS)
        CPU: 4.596s

Oct 28 02:54:37 proxmox6 pveupdate[1526907]: <root@pam> end task UPID:proxmox6:00174C97:9CCC761B:635B8AB8:aptupdate::root@pam: OK
Oct 28 02:54:37 proxmox6 pveupdate[1526907]: CN = Proxmox Virtual Environment, OU = 7ab84c1f25372786d53d32760a492a5e, O = PVE Cluster Manager CA
Oct 28 02:54:37 proxmox6 pveupdate[1526907]: error 10 at 1 depth lookup: certificate has expired
Oct 28 02:54:37 proxmox6 pveupdate[1526907]: OU = PVE Cluster Node, O = Proxmox Virtual Environment, CN = proxmox6.telecore.local
Oct 28 02:54:37 proxmox6 pveupdate[1526907]: error 10 at 0 depth lookup: certificate has expired
Oct 28 02:54:37 proxmox6 pveupdate[1526907]: error /etc/pve/nodes/proxmox6/pve-ssl.pem: verification failed
Oct 28 02:54:37 proxmox6 pveupdate[1526907]: Checking/Renewing SSL certificate failed: command '/usr/bin/openssl verify -CAfile /etc/pve/pve-root-ca.pem /etc/pve/nodes/proxmox6/pve-ssl.pem' failed: exit code 2
Oct 28 02:54:37 proxmox6 systemd[1]: pve-daily-update.service: Succeeded.
Oct 28 02:54:37 proxmox6 systemd[1]: Finished Daily PVE download activities.
Oct 28 02:54:37 proxmox6 systemd[1]: pve-daily-update.service: Consumed 4.596s CPU time.

 

[Zdenek Stepanek]

Same problem to me.

Attempt to add node to cluster. All nodes are fully updated to 8.1.4.

Establishing API connection with host '10.193.65.10'
Login succeeded.

check cluster join API version
No cluster network links passed explicitly, fallback to local node IP '10.193.65.35'
Request addition of this node
Join request OK, finishing setup locally
stopping pve-cluster service
backup old database to '/var/lib/pve-cluster/backup/config-1706803133.sql.gz'
waiting for quorum...OK
(re)generate node files
generate new node certificate
TASK ERROR: CA expires in less than 2 weeks, unable to generate certificate.

Altough new node is added to left panel (tree) and is green, but then is WebUI full with "error:0A000086:SSL routines::certificate verify failed (596)" error messages.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This was happen due to CA certificate is too old. It was generated for 10 years and if you have such old cluster, then you have problem.

What helped to me:

On one node - "/etc/pve" is shared filesystem.

cd /root
mkdir oldkeys

mv /etc/pve/pve-root-ca.pem oldkeys
mv /etc/pve/priv/pve-root-ca.key oldkeys
mv /etc/pve/authkey.pub oldkeys
mv /etc/pve/priv/authkey.key oldkeys
mv /etc/pve/priv/authorized_keys oldkeys

And for EACH node in cluster:

mv /etc/pve/nodes/NODENAME/pve-ssl.pem oldkeys
mv /etc/pve/nodes/NODENAME/pve-ssl.key oldkeys

Then run these two commands on EACH node. On first node, because you deleted expired CA, it will create new global CA and also key for local node. Then run them also on all remains nodes. It will find new CA and will generate key for local node only.

pvecm updatecerts -f
systemctl restart pvedaemon pveproxy

 

[esi_y]

Filed: https://bugzilla.proxmox.com/show_bug.cgi?id=5215