Certificate Validation Error Upon Trying to Install License

[dtheese]

Upon attempting to install a Proxmox VE license, I get the following error:

"Invalid response from server: 500 Can't connect to shop.maurer-it.com:443 (certificate verify failed) (500)"
I've attempted working with Proxmox tech support, but they've been quite unhelpful.

My organization proxies connections on port 443 with a middlebox. I suspect, but do not know, that the problem lies here.

When the Proxmox license-verifying client reaches out to shop.maurer-it.com to verify the license I've entered, it receives a certificate issued by my organization's private CA. This is a perfectly valid certificate with a CN of "shop.maurer-it.com", a valid expiration date, a valid "not good before" date, etc. I've got our middlebox's CA certificates installed in OpenSSL (and I did manage to find out from Proxmox tech support that it is the OpenSSL certificate store they use), and I can successfully use OpenSSL to verify certificates issued by our proxy.

I can even use wget to retrieve resources such as https://shop.maurer-it.com/index.php.

My suspicion is that the Proxmox client program that validates Proxmox license keys is using public key pinning (or some other, perhaps proprietary, mechanism). But this is only a suspicion. I've so far been unable to think of anything else that might cause this.

Can anybody confirm or deny if the Proxmox client program that validates license keys uses public key pinning (or some equivalent mechanism)?

What other issues that I've failed to consider could cause the problem I'm experiencing?

In the meantime, I've put in a request with out IT department to whitelist shop.maurer-it.com:443 so that the proxy is bypassed and I get the real certificate for shop.maurer-it.com. This will answer the question for sure. But this takes time since IT has to do a risk assessment. So, I am making inquiries of tech support and of the community to see if I should even be spending my time on this as a potential source of the problem...

Thank you in advance for any information provided.

 

[tom]

Am I right that your proxy is doing a man-in-the-middle attack into https traffic and replaces our certificate.?

 

[dtheese]

Yes, our proxy is a MITM that replaces your certificate (though it is done legitimately, not as an attack). The certificate the proxy provides is perfectly valid in terms of all of the usual checks done to validate a certificate. The question is whether or not your client program that reaches out to shop.maurer-it.com to verify license keys will work with such a certificate. Is it checking that the value of the public key is as expected (i.e. public key pinning), is it checking that the issuer is as expected (Let's Encrypt vs. our middlebox), etc.??? If it's making some sort of a check for a MITM and will refuse to operate under those circumstances, that tells me I need to keep pursuing having our IT department whitelist shop.maurer-it.com. But if not, I need to focus my efforts elsewhere...

 

[fabian]

no, there are no such checks in our code (it should rely on the stock ca-certificates and SAN/CN matching).

please double check you have correctly installed your MITM-box' CA/root certificate on the PVE host (see "man update-ca-certificates", it does not need to be stored in /etc/ssl/certs, but it needs to be correctly part of the file ca-certificates.crt which is generated by update-ca-certificates!), and that the hostname in the generated certificate is really correct. also note that the request is a POST not a GET, might make a difference for your box.

 

[dtheese]

Ahhh, there we go.

I had the hash symlinks in /etc/ssl/certs (irrelevant for this case, but needed for unrelated HTTPS operations), but update-ca-certificates had not put the certs in ca-certificates.txt. This occurred because I missed the requirement that the certificates be placed in /usr/local/share/ca-certificates with a *.crt* extension. (The certs are stored as *.pem* files in /share/CA_Certs. I have symlinks to the certs in /usr/local/share/ca-certificates. But I had the symlinks named with .pem since they're pointing to .pem. Changing the link names to .crt fixed the problem.)

I have successfully installed the license, and, after shutting down all VMs / containers, I have successfully performed an apt-get update and an apt-get upgrade against the Proxmox enterprise repository (and, of course, the Debian repositories).

Upon reboot, I found that the containers would start but that my two Debian VMs would not.

In any case, that's a separate issue. This thread's issue has been addressed. I don't see a button to explicitly close the thread, but I will "Like" your response and consider the thread closed. I will also close the corresponding ticket on the customer portal.

If I can't solve the issue of the VMs not booting, I'll open another forum thread. I won't open a ticket on the customer portal because this now goes beyond the scope of a community license--installing the license and upgrading against the enterprise repository.

Thank you *very much* for your assistance!

 

[tom]

apt-get upgrade

never do this, this can break your system.

please follow the upgrade guides on:
https://pve.proxmox.com/wiki/Downlo...Proxmox_Virtual_Environment_5.x_to_latest_5.1