ACME Order "Failed - server reply does not look like a PEM encoded certificate"

G

gmbakken

New Member
Oct 21, 2024
5
1
3
Hello! I am trying to get our ACME server running Keyfactor EJBCA to work with Proxmox nodes. However, at the end of the order I receive an error stating that the encoding is wrong. I cannot find others with the same issue, we have this issue on multiple nodes and different environments. We are using HTTP challenge. Maybe someone here with experience?

BTW I am new to Proxmox.


All domains validated!

Creating CSR
Checking order status
Order is ready, finalizing order
valid!

Downloading certificate
TASK ERROR: POST of 'https://*****.net/ejbca/acme/*****/cert/************0e2f6e' failed - Server reply does not look like a PEM encoded certificate
 
Last edited:
  • Like
Reactions: silkeslips
Bumping the topic.
Having the same issue here.

Code: 
Loading ACME account details
Placing ACME order
Order URL: https://<pki_provider>/ejbca/acme/G-Acme/acct/jGiid*****roi5A/orders/DFSYq4T*****4xd_JTFUc

Getting authorization details from 'https://<pki_provider>/ejbca/acme/G-Acme/acct/jGiid*****roi5A/authz/9b3cc3*****9d273b35b8'
The validation for ci-3d-pve1-1.<domain_name> is pending!
Setting up webserver
Triggering validation
Sleeping for 5 seconds
Status is 'valid', domain 'ci-3d-pve1-1.<domain_name>' OK!

All domains validated!

Creating CSR
Checking order status
Order is ready, finalizing order
valid!

Downloading certificate
TASK ERROR: POST of 'https://<pki_provider>/ejbca/acme/G-Acme/cert/e0f319******5186' failed - Server reply does not look like a PEM encoded certificate

On the ACME server side, the certificates have been successfully generated and it views them as in use. The server setup is likely correct as all other clients using this server have no issues.

Could this be a Proxmox bug?
 
Still having this problem. I had a look at the EJBCA log.

HTTP: 
Response status: 200
ContentType: application/pem-certificate-chain;charset=utf-8
CharacterEncoding: utf-8
(Not yet final) Response headers:
 Replay-Nonce ->
  Replay-Nonce: AAABk6bI3....HhESgEiQsFGw
 Link ->
  Link: <https://xxx/ejbca/acme/directory>; rel="index"
 Content-Type ->
  Content-Type: application/pem-certificate-chain;charset=utf-8
Response data:
    Subject: CN=xxxx,OU=...
Issuer: CN=Issuing CA,O=....
-----BEGIN CERTIFICATE-----
MIIIyT...

From my perspective the 2 lines before the PEM header might be the problem.

Does anybody know how to prevent that EJBCA sends this two lines or how to configure proxmox to ignore them?
 
We have a support ticket with both Proxmox and Keyfactor regarding this at the moment.

So far it seems like Proxmox does have problems with data before the encapsulation boundaries in the PEM file. The RFC defining this makes it clear that this is permitted and we are awaiting their response on getting a fix for this.

On the EJBCA side, we have not found any way to turn the extra lines off. Our support ticket is still open but we have not yet gotten a conclusive answer on it.
 
We now have a response from both Proxmox and Keyfactor.

Proxmox responded saying that the RFC for ACME certificates specifies that the extra data is not to be included, even if it is generally allowed. They also said that this does not mean they might not relax the parsing, and have opened a enhancement request in their bugtracker.

Keyfactor agrees that they should not include the extra data, and have opened a ticket on their side to investigate this further (Ref: ECA-12860).
 
  • Like
Reactions: gmbakken
Great thanks,
On our end with are using a service called Venafi to play the ACME server role. If we use the pvenode acme commands, or with the proxmox gui i have the exact same error as you.

So i downloaded the acme.sh script on the proxmox box, just to test out if that works. This works perfectly, acme.sh does everything fine and does get the certificate. So for my part, the ACME server is fine, since it's working with the acme.sh script. From what i understand, in my perception, it's the proxmox implementation of acme.sh that is causing problems.
 
Recently ran into this issue as well, with Proxmox's ACME client trying to order a certificate from our internal ACME server and failing. The domain validation worked fine, but when trying to download the certificate, it printed out:

Bash: 
Downloading certificate
POST of 'https://acme.internal/acme/cert/UNIQCERTID' failed - Server reply does not look like a PEM encoded certificate
Task POST of 'https://acme.internal/acme/cert/UNIQCERTID' failed - Server reply does not look like a PEM encoded certificate

After digging into the code, I found the issue is with the regex used to valdiate the certificat response inside the PVE::ACME module.
My specific error seem to errupt from 2-3 additional new lines after the certificate chain of 3 certificates. Copying the complete response and Regex into regexr, it does not validate, but removing the extra 2-3 lines at the bottom makes it validate. The regex seems overly strict in that it doesn't allow anything additional either before or after the certificate chain. It either "approves" of the response as-a-whole, or discards all of it.

Here is an updated version, which grabs the PEM cerificate chain, and ignores everything else:
Diff: 
--- /usr/share/perl5/PVE/ACME.pm (original)
+++ /usr/share/perl5/PVE/ACME.pm (fixed)
@@ -536,8 +536,8 @@
-    if ($res =~ /^(-----BEGIN CERTIFICATE-----)(.+)(-----END CERTIFICATE-----)$/s) { # untaint
-        return $1 . $2 . $3;
+    if ($res =~ /((?:-----BEGIN CERTIFICATE-----.*?-----END CERTIFICATE-----\s*)+)/s) { # untaint
+        return $1;
     }
The change is easiest to do manually, no reboot required.

I have verified this change makes pvenode acme cert order work with our internal ACME server. I've updated the Bugzilla ticket as well.
 
You must log in or register to reply here.
Top Bottom