One LetsEncrypt-certificate for all Clustermember and access behind an haproxy

udo

Distinguished Member
Apr 22, 2009
5,975
196
163
Ahrensburg; Germany
Hi,
I have just install on my home-cluster letsencrypt with the same certificate for all clustermember.
Pro:
- I need recreate one certificate only. (not an big plus)
- I can access the cluster from outside with one address through haproxy (Port 8006) and have an valid certificate equal on which node I logged in. (an real benefit!)

The cert combine all nodename plus the "default-landing-name" for extern:
Code:
Certificate Subject Alt Name:
DNS Name: pve-a.example.com
DNS Name: pve-b.example.com
DNS Name: pve-c.example.com
DNS Name: pve.example.com
If someone interesting in an tutorial I can posted it here.

Udo
 
> If someone interesting in an tutorial I can posted it here.

Yes, its's very cool!
 
Partly covered from german howto https://www.deimeke.net/dirk/blog/index.php?/plugin/tag/letsencrypt

The installation is needed on one cluster-node only - in this example the nodes are pve-a, pve-b and pve-c.example.com .
The first nodes is reachable as 192.168.200.11

The DNS/Firewall must be configured, that pve.example.com, pve-a.example.com, pve-b.exapmle.com and pve-c.example.com are directed to the proxy-server with haproxy (cnames for dns).

Important 1! If you allready used letsencrypt with the "default" method it's very important to use the account.key from /etc/pve/.le/
Otherwise you are not able to get certificates with the same name as allready used!

Important 2!
run an backup of /etc/pve, like:
Code:
tar cvf /root/etc_pve.tar /etc/pve/

Preparations
Code:
# not nesseary if /etc/pve/.le/account.key excist!
mkdir /etc/pve/.le/
openssl genrsa 4096 > /etc/pve/.le/letsencrypt-pve.key
If you used letsencryt before, disable/delete the cronjob (crontab -e) about /root/.acme.sh !
To be sure, move the dir:
Code:
mv /root/.acme.sh /root/.acme.sh.old
Install needed tools:
Code:
apt install libssl-dev gcc unzip make
get AcmeFetch and install
Code:
# download the latest version https://github.com/oetiker/AcmeFetch/releases/latest

wget https://github.com/oetiker/AcmeFetch/releases/download/v0.5.0/acmefetch-0.5.0.tar.gz
tar xvzf acmefetch-0.5.0.tar.gz
cd acmefetch-0.5.0/
./configure --prefix=/opt/acmefetch
/usr/bin/make install
for security remove gcc+make again
Code:
apt remove gcc make
steps for temporary webserver
Code:
mkdir -p /opt/acmefetch/www/.well-known/acme-challenge

cat > /opt/acmefetch/www/index.html << EOF
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html><head>
<meta content="text/html; charset=ISO-8859-1" http-equiv="content-type">
<title>temporary website</title>
<meta content="Author" name="pve"></head>
<body>Hello.<br>
The python httpd server is working.<br>
and waiting for letsencrypt<br>
</body></html>
EOF
create script to create/renew cert:
Code:
cat > /usr/local/sbin/le_cert_renew.sh << EOF
#!/bin/bash
#
# le_cert_renew.sh renew certificates if nessesary
# (ul) 20161029 first version

# start simple http-server
cd /opt/acmefetch/www
python3 -m http.server 8080&
http_pid=\$(ps aux | grep "python3 -m http.server 8080" | grep -v grep | awk '{ print \$2 }')


/opt/acmefetch/bin/acmefetch --cfg=/opt/acmefetch/etc/acmefetch.cfg

# stop http-server
kill \$http_pid

# at the first run pveproxy-ssl.key don't excist
if [[ ! -e /etc/pve/local/pveproxy-ssl.key ]]
then
  touch /etc/pve/local/pveproxy-ssl.key
fi

if [[ \$(diff -q /etc/ssl/private/pveproxy-ssl.key /etc/pve/local/pveproxy-ssl.key) ]]
then
  nodes=\$(ls /etc/pve/nodes)
  for i in \$nodes
  do
  cp -p /etc/ssl/private/pveproxy-ssl.key /etc/pve/nodes/\$i/
  cat /etc/ssl/certs/pveproxy-ssl.pem > /etc/pve/nodes/\$i/pveproxy-ssl.pem
  cat /etc/ssl/certs/pveproxy-ssl-chain.pem >> /etc/pve/nodes/\$i/pveproxy-ssl.pem
  ssh \$i service pveproxy restart
  done
fi
EOF

chmod +x /usr/local/sbin/le_cert_renew.sh
create and modify acmefetch config to fit for your systems (accountKeyPath + commonName + sites)
- the forum eat spaces, which are important for the config-file.
create the file first and after that edit the file for the right values:
Code:
cat /opt/acmefetch/etc/acmefetch.cfg
{
  "GENERAL": {
   "ACMEstaging": "acme-staging.api.letsencrypt.org",
   "ACMEservice": "acme-v01.api.letsencrypt.org",
   "accountKeyPath": "/etc/pve/.le/account.key"
  },
   "CERTS": [
       {
        "certOutput": "/etc/ssl/certs/pveproxy-ssl.pem",
        "certFormat": "PEM",
        "keyOutput": "/etc/ssl/private/pveproxy-ssl.key",
        "keyFormat": "PEM",
        "chainOutput": "/etc/ssl/certs/pveproxy-ssl-chain.pem",
        "chainFormat": "PEM",
        "commonName": "pve.example.com",
        "SITES": {
           "pve.example.com": {
              "challengeHandler": "LocalFile",
              "challengeConfig": {
              "www_root": "/opt/acmefetch/www/",
              }
          },
              "pve-a.example.com": {
                "challengeHandler": "LocalFile",
                "challengeConfig": {
                "www_root": "/opt/acmefetch/www/",
                }
              },
              "pve-b.example.com": {
                "challengeHandler": "LocalFile",
                "challengeConfig": {
                "www_root": "/opt/acmefetch/www/",
                 }
              },
               "pve-c.example.com": {
                  "challengeHandler": "LocalFile",
                 "challengeConfig": {
                 "www_root": "/opt/acmefetch/www/",
                 }
              },
          }
      }
  ]
}
extend HAproxy config on your haproxy-server - first the part for LetsEncrypt:
Code:
frontend http_frontend
  bind *:80
  mode http
  acl acl_pve  hdr(host) -i pve.example.com
  acl acl_pve  hdr(host) -i pve-a.example.com
  acl acl_pve  hdr(host) -i pve-b.example.com
  acl acl_pve  hdr(host) -i pve-c.example.com
  use_backend pve-http  if acl_pve

backend pve-http
  mode http
  server pve 192.168.200.11:8080
Code:
service haproxy reload
On the first node start the webserver to control the external access
Code:
cd /opt/acmefetch/www
python3 -m http.server 8080
from external must be following appear in an browser with pve.example.com, pve-a.example.com and so on:
Code:
  Hello.
  The python httpd server is working.
  and waiting for letsencrypt
If all right stop the web-server with <cntl>-C

Lets start
Code:
/usr/local/sbin/le_cert_renew.sh
 
Last edited:
To access the pve-gui from outside use a config like this in haproxy.cfg on your proxy-host:
Code:
frontend pve_frontend
  bind *:8006
  mode tcp
  option httpclose
  option forwardfor
  reqadd X-Forwarded-Proto:\ https
  default_backend pve-server
  timeout client  65000

backend pve-server
  mode tcp
  balance roundrobin
  timeout server  65000
  stick-table type ip size 200k expire 60m
  stick on src
  server pve-a 192.168.200.11:8006
  server pve-b 192.168.200.12:8006
  server pve-c 192.168.200.13:8006
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!