[SOLVED] Setting up cluster and certificates: which order?

[tobox]

I am new to proxmox and I have just set up my first 2 nodes. It replaced the certificates in

/etc/pve/nodes/pve1/pve-ssl.pem
/etc/pve/nodes/pve1/pve-ssl.key
and
/etc/pve/nodes/pve2/pve-ssl.pem
/etc/pve/nodes/pve2/pve-ssl.key

by trusted certificates, and then I created a new cluster on pve1. That worked without problems.

Then I tried to join that cluster from the web interface of pve2, and the following happened:

- the web interface became grey and a message similar to "permission denied invalid pve ticket" appeared in the background
- pve2 seemed to have been added to the new cluster successfully
- pve2 now has some kind of invalid http certificate

What's the suggested way of setting up a cluster and using own certificates? Create the cluster first (with self signed certs) and update the certs later on? How would I add an additional node later on, if the other nodes are already using proper certs?

I hope those are not stupid newbie questions, but the documentation was not very clear on this....

Regards
Thomas

 

[Stoiko Ivanov]

Our reference documentation describes this procedure pretty well - https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#_certificates_for_api_and_web_gui

Put shortly - you replaced the cluster-communication certificates (not the optional pveproxy/webgui certificates)

running pvecm updatecerts --force should get the cluster operational again (save your trusted certificates somewhere else, they will get overwritten!)

I hope this helps!

 

[tobox]

Thanks for the reply. I read the documentation but I misinterpreted the following part:

Do not replace or manually modify the automatically generated node certificate files in /etc/pve/local/pve-ssl.pem and /etc/pve/local/pve-ssl.key or the cluster CA files in /etc/pve/pve-root-ca.pem and /etc/pve/priv/pve-root-ca.key.

I thought I was allowed to replace all other certificates, except for the ones listet as "do not replace...".

But now everything is clear. Thanks!

 

[Stoiko Ivanov]

I thought I was allowed to replace all other certificates, except for the ones listet as "do not replace...".

Hmm - quite understandable - we probably should explicitly mention that '/etc/pve/local/' is a symlink to '/etc/pve/NODENAME/'

Thanks for the hint!

Please mark the thread as 'SOLVED' - this helps others with similar problems

 

[tobox]

Now I am confused... Where exactly do I put my externally created certificate files? Or should I not mess with any of the files directly?

The documentation only mentions letsencrypt ACME, but not externally generated certificates.

 

[Stoiko Ivanov]

hm?
the docs say:

You have the following options for the certificate used by pveproxy:

1. By default the node-specific certificate in /etc/pve/nodes/NODENAME/pve-ssl.pem is used. This certificate is signed by the cluster CA and therefore not trusted by browsers and operating systems by default.
2. use an externally provided certificate (e.g. signed by a commercial CA).
3. use ACME (e.g., Let’s Encrypt) to get a trusted certificate with automatic renewal.

For options 2 and 3 the file /etc/pve/local/pveproxy-ssl.pem (and /etc/pve/local/pveproxy-ssl.key, which needs to be without password) is used.
Certificates are managed with the Proxmox VE Node management command (see the pvenode(1) manpage).

you want to use an external cert (point 2 above), in which case you need to put the private key in `/etc/pve/local/pveproxy-ssl.key` and the certificate in `/etc/pve/local/pveproxy-ssl.pem`

I hope this explains it!

 

[tobox]

I was confused by the symlink between /etc/pve/local and /etc/pve/nodes/HOSTNAME... My fault was that I replaced the existing pve-ssl.{key,pem} files instead of creating new files pveproxy-ssl.{key,pem}.

Solved!