Google Domains and Let's Encrypt Certificates using DNS validation for local Proxmox servers

[cocoboig]

I'd like share my experience with Google Domains and Let's Encrypt Certificates (ACME protocol) for local proxmox servers using DNS validation. You are free to modify and improve this guide.

Sometimes you can prefer get Let's Encrypt Certificates using DNS validation instead of HTTP protocol, for example, you don't want to open ports in your local network. If you have a domain with Google Domains and you are using their nameservers to configure your domain you don't have an API. But you can delegate your local zone to other nameservers.

The example for this guide is going to use Google Cloud DNS service to manage delegated local zone. Requirements:

Your domain: example.com
Your local domain zone (subdomain of example.com): home.example.com
Your server (FQDN - full qualified domain): proxmox.home.example.com
You have a local DNS server for the zone home.example.com
We are using an external DNS server for DNS validation with ACME protocol, in our example, Google Cloud DNS.


.- Go to Google Cloud DNS and add a new project, enable DNS API and create a DNS zone (see https://cloud.google.com/dns/docs/quickstart). After you have created the zone you have two records that look as:

home.example.cc. SOA 21600 ns-cloud-e1.googledomains.com. cloud-dns-hostmaster.google.com. 46 21600 3600 259200 300
home.comas.cc. NS 21600
ns-cloud-e1.googledomains.com.
ns-cloud-e2.googledomains.com.
ns-cloud-e3.googledomains.com.
ns-cloud-e4.googledomains.com.

ns-cloud-XX.googledomains.com. can be different
(quick link to zones: https://console.cloud.google.com/net-services/dns/zones/)


.- Go to Google Domains and add following records to example.com domain:

_acme-challenge.proxmox.home CNAME 1h home.example.com
home NS 6h ns-cloud-e1.googledomains.com.
ns-cloud-e2.googledomains.com.
ns-cloud-e3.googledomains.com.
ns-cloud-e4.googledomains.com.

Use ns-cloud-XX.googledomains.com. from Google Cloud DNS assigned to your zone.


.- Open Proxmox gui and click on "Datacenter->ACME" and add new account using Let's Encrypt V2 (if you want test it then I recommend that you choose Let's Encrypt V2 Staging). At same screen click on "Add" button of "Challenge Plugins" and type:
Plugin ID: gcloud
DNS API: gcloud
API Data: HOME=/home/nobody


.- Go to Proxmox shell and install Google Cloud SDK:

# Add the Cloud SDK distribution URI as a package source
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list

# Import the Google Cloud Platform public key
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -

# Update the package list and install the Cloud SDK
apt-get update && apt-get install google-cloud-sdk

# Init Google Cloud (gcloud) access and select project
gcloud init


.- Replicate configuration for nobody user

# Create /home/nobody directory
mkdir /home/nobody

# Create /home/.config directory
cd /home/nobody
mkdir .config

# Copy Google Cloud settings
cd .config
cp -pr /root/.config/gcloud/ .

# Set nobody user and nogroup group to /home/nobody recursively
chown -R nobody:nogroup /home/nobody


.- Add validation-delay parameter to gcloud plugin

pvenode acme plugin set gcloud --validation-delay 120
(acme gcloud dns plugin use 30 seconds but sometimes it isn't enough for TXT record replication to nameservers)


.- Edit /usr/share/perl5/PVE/ACME/DNSChallenge.pm or wait new DNSChallenge.pm version (see https://git.proxmox.com/?p=proxmox-acme.git;a=commitdiff;h=1192b595869adcf12eda5dc0cf7db950625a22b6)

Replace
my $delay = $data->{'validation-delay'} // 30;
by
my $delay = $data->{plugin}->{'validation-delay'} // 30;

.- Order certificates
pvenode acme cert order

.- If you need check all gcloud dns tasks are done:
gcloud dns record-sets changes list --zone="myzonename"


Regards,
Josep Comas


References:

.- More info about Certificate Management in Proxmox:
https://pve.proxmox.com/wiki/Certificate_Management

.- pvenode command and acme options:
https://pve.proxmox.com/pve-docs/pvenode.1.html

.- If you want build your own gcloud dns plugin or test all is working:
https://cloud.google.com/sdk/gcloud/reference/dns

.- See your DNS zones for your current project:
https://console.cloud.google.com/net-services/dns/zones

.- Let's encrypt limits:
Production environment: https://letsencrypt.org/docs/rate-limits/
Test environment: https://letsencrypt.org/docs/staging-environment/

.- You have "dig" and "nslookup" to check DNS records, if you prefer to check online:
https://dns.google.com/

.- If you need alternative configuration for gcloud dns acme plugin see:
https://github.com/acmesh-official/...gle-cloud-dns-api-to-automatically-issue-cert

.- Diagnostic tool for Let's Encrypt:
https://letsdebug.net/

.- Security considerations on ACME DNS validation:
https://www.eff.org/deeplinks/2018/...ring-automation-acme-dns-challenge-validation

.- Inspiration for this guide:
https://certhub.readthedocs.io/en/latest/practice/dns.html

.- Google Domains:
https://domains.google.com/m/registrar/

.- Additional reading:
http://sebastian.korotkiewicz.eu/te...ncrypt-wildcard-certificates-with-local-bind/

.- To check DNS propagation:
https://www.whatsmydns.net/ (type home.example.com and select NS record type)


To do:
.- Add screenshots
.- All command line

 

[jlamoree]

I wanted to setup a similar configuration yesterday. I recorded a video showing how to setup Proxmox VE 6.2 with Let's Encrypt using ACME DNS plugin challenge verification using Amazon Route 53. https://vimeo.com/422553702

I'm very new to the Proxmox VE product, so I don't know what is normal for development. I assume that features are added to CLI tools like `pvenode` and eventual given a web UI for the most common uses.

 

[roaman]

Thanks for this helpful post, it saved me a lot of time figuring out this pretty odd workaround. I used the same method to get the gcloud dns plugin working on Proxmox Backup Server and Proxmox Mail Gateway too.

In my case I ended up actually having to make a link to the "nobody" home directory at /nonexistent as well, seems without that it wouldn't take. And in case this helps anyone else out in the future, I use a service account to authenticate to gcloud for the zone updates, and in addition to needing credentials file in /home/nobody this also required running one last additional command as nobody to active the service account. So:

mkdir -p /home/nobody/.config
cp -r ~/.config/gcloud /home/nobody/.config/
cp dns-google-creds-mine.json /home/nobody/
chown -R nobody:nogroup /home/nobody/

ln -s /home/nobody /nonexistent
# to activate service account:
setpriv --reuid nobody --regid nogroup --clear-groups --reset-env -- /usr/bin/gcloud auth activate-service-account service-name@gcloud-address.com --key-file=/home/nobody/dns-google-creds-mine.json

 

[stackofone]

Wanted to share this was super helpful and pointed me in the right direction, but I was stubborn and didn't want to copy config files.

I'm pretty sure you still need to create the /home/nobody/.config and copy the service account key to /home/nobody/ folder, but instead of copying the gcloud config folder, you can add the following to API Data in the Datacenter > ACME Plugin Config

HOME=/home/nobody
CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE=/home/nobody/<your-service-account-key>.json
CLOUDSDK_CORE_PROJECT=<your-project-name>

PS: Also, helpful tip for those obsesses with least privileges for GCP IAM, the service account needs the `DNS Reader` role at the project level.