Broken Proxmox by trying to add SSL Certificates

Aug 6, 2019
22
1
8
53
I recently made a complete mess of my Proxmox server (it's a homelab) by trying to install a GPU. Made such a mess that I gave up and reinstalled everything. This was fine until I tried to install the self-signed certificates that I'd had installed on the previous version of the server. (They were created with the Docker container division42llc/dotnet-webca)

I obviously did something wrong because now I can't access the web interface. Chrome and Firefox tell me the server closed the connection and for some reason Safari tells me I'm not connected to the internet.

I followed the instructions here

https://pve.proxmox.com/wiki/HTTPS_Certificate_Configuration_(Version_4.x,_5.0_and_5.1)

because I can't seem to find the 6.0 version with various searches. I did a search for .pem and .key and .ca and removed everything, restarted the server a few times but I'm in the same position. The server is working - all the VMs are up. I just can't get to the web interface.

Can someone tell me what else I need to delete and/or point me to the 6.0 version of the above wiki document?

Thanks,

Steve
 
That document is no use at all. The very first paragraph tells you it's not what I'm looking for. And to offer "I know this does not work at the moment, but if it would you could." as a suggestion makes me wonder what I pay my subscription for.
 
That document is no use at all. The very first paragraph tells you it's not what I'm looking for. And to offer "I know this does not work at the moment, but if it would you could." as a suggestion makes me wonder what I pay my subscription for.

that document is exactly what you should use - it migh thelp to read more than the first paragraph..

The REST API and web GUI are provided by the pveproxy service, which runs on each node.
You have the following options for the certificate used by pveproxy:
  1. By default the node-specific certificate in /etc/pve/nodes/NODENAME/pve-ssl.pem is used. This certificate is signed by the cluster CA and therefore not trusted by browsers and operating systems by default.
  2. use an externally provided certificate (e.g. signed by a commercial CA).
  3. use ACME (e.g., Let’s Encrypt) to get a trusted certificate with automatic renewal.
For options 2 and 3 the file /etc/pve/local/pveproxy-ssl.pem (and /etc/pve/local/pveproxy-ssl.key, which needs to be without password) is used.
Certificates are managed with the Proxmox VE Node management command (see the pvenode(1) manpage).

Do not replace or manually modify the automatically generated node certificate files in /etc/pve/local/pve-ssl.pem and /etc/pve/local/pve-ssl.key or the cluster CA files in /etc/pve/pve-root-ca.pem and /etc/pve/priv/pve-root-ca.key.

I suggest the following:
- remove /etc/pve/priv/pve-root-ca.key
- remove /etc/pve/pve-root-ca.pem
- remove /etc/pve/local/pve-ssl.pem /etc/pve/local/pve-ssl.key /etc/pve/local/pveproxy-ssl.pem /etc/pve/local/pveproxy-ssl.key (on each node!)
- run 'pvecm updatecerts'
- run 'systemctl restart pveproxy'

now the web interface should work again, with the default self-signed certificates. now use the GUI or 'pvenode' to upload your certificate+key (it will be stored in '/etc/pve/local/pveproxy-ssl.pem' / '/etc/pve/local/pveproxy-ssl.key'
 
  • Like
Reactions: jtremblay and LinFJ
If you have a subscription with basic, standard or premium level you can contact the Support via Customer Portal.
At the customer portal your setup will be inspected in detail.
If you have a community-level subscription the support does not include a detailed failure analysis.
That document is no use at all.
See Fabian's answer.
 
- remove /etc/pve/priv/pve-root-ca.key
- remove /etc/pve/pve-root-ca.pem
- remove /etc/pve/local/pve-ssl.pem /etc/pve/local/pve-ssl.key /etc/pve/local/pveproxy-ssl.pem /etc/pve/local/pveproxy-ssl.key (on each node!)
- run 'pvecm updatecerts'
- run 'systemctl restart pveproxy'
Thanks, fabian, very much! That saved my day after having destroyed my SSL/HTTPS connectivity by trying to upload my own certificates via WEB-GUI.
 
I'm also struggling to setup my certificate, so that there is ever only 1 private key to use for all my services. I would like to retrieve the cert once and then distribute it to all VMs and LXCs. In contrast to OP, I would like to have a wildcard cert, but the only difference here is the "Subject Alternative Name".

First I tried to issue a wildcard cert directly with the acme support in "Datacenter" but is sadly enough not supported... (Another Thread discussion)
Single domain works just perfect.

Next thing I tried is to upload a custom cert, as OP tired to do.
I issued a cert with certbot
Bash:
certbot certonly --manual --preferred-challenges=dns --email admin@example.com --server https://acme-v02.api.letsencrypt.org/directory --agree-tos -d *.example.com -d example.com
and got 4 files:
  • cert.pem
  • chain.pem
  • fullchain.pem (cert + chain)
  • privkey.pem
I uploaded fullchain and privkey and got the same problem as OP.

I compared the output of the proxmox acme cert (pveproxy-ssl.pem) and my certbot cert (fullchain.pem), the only difference being a missing new line between "cert" and "chain".

Code:
-----BEGIN CERTIFICATE-----
domain pubkey
-----END CERTIFICATE-----
******* MISSING NEW LINE *******
-----BEGIN CERTIFICATE-----
chain form let's encrypt
-----END CERTIFICATE-----

-----BEGIN CERTIFICATE-----
chain form let's encrypt
-----END CERTIFICATE-----

I'm not sure about specification if there needs to be a new line (=> certbot "bug") or if it can be ommitted (=> proxmox "bug").
Apache seems to not have no problem with the missing new line.

Wildcard support would be really nice to have.
 
does PVE accept the chain if you add the new line to the one generated by certbot?
 
then that's definitely something that should be improved! could you file an issue at https://bugzilla.proxmox.com with details (installed version, if possible the chain or the chain with just the server/leaf cert censored, ..)? thanks!
 
that document is exactly what you should use - it migh thelp to read more than the first paragraph..



I suggest the following:
- remove /etc/pve/priv/pve-root-ca.key
- remove /etc/pve/pve-root-ca.pem
- remove /etc/pve/local/pve-ssl.pem /etc/pve/local/pve-ssl.key /etc/pve/local/pveproxy-ssl.pem /etc/pve/local/pveproxy-ssl.key (on each node!)
- run 'pvecm updatecerts'
- run 'systemctl restart pveproxy'

now the web interface should work again, with the default self-signed certificates. now use the GUI or 'pvenode' to upload your certificate+key (it will be stored in '/etc/pve/local/pveproxy-ssl.pem' / '/etc/pve/local/pveproxy-ssl.key'
I tried to import Certs I created in FreeIPA and borked the WebUI THIS SOLVED THE ISSUE! TY
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!