VNC TLS handshake failed

[wawjohn]

I've got this to work reliably.

The important thing is to not touch the certificates and keys in /etc/pve, especially not pve-root-ca.pem.

I simply copied my official purchased SSL certificate and intermediate certificate (I use RapidSSL), plus the certificate private key into the usual places, (/etc/ssl/certs and /etc/ssl/private respectively.

I then configured the files in /etc/apache2/sites-enabled (actually links to files in sites-available) pve.conf and pve-redirect.conf to use the SSL files in /etc/ssl instead of the files in /etc/pve:

SSLCertificateFile /etc/ssl/certs/server.example.com.pem
SSLCertificateKeyFile /etc/ssl/private/server.example.com.key
SSLCertificateChainFile /etc/ssl/certs/RapidSSL-intermediate-CA.pem

After reboot, I can connect to https://my-proxmox.example.com:8006 and see the correct official certificate.

When I use the console VNC java applet, I still get asked to accept the certificate and the fact that the applet has full access to my system and that I understand the risks (browser: Safari on Mac OS X Lion), but once I acquiesce, the VNC console works as expected.

This is where I found the key to this solution:

An email from Kurt Smolderen on the pve-devel list. Thanks Kurt!

 

[degrootm]

Thanks $1M wawjohn, it took me a while to get back to this but I really needed to fix a VM that had a configuration issue. So thank goodness you left your note.
I took the liberty of rewriting your notes and others into one collective step-by-step. To perhaps save some others from sliding off the edge as I had. As we know not everything you read on the internet is always current or true. So This is specifcally for the current version of ProxMox VE: 2.2-32


Step-by-step:


0. Backup your SSL files on all cluster nodes. Just in case you go mad.
Since you should never need to touch them.
minimum:
# cp /etc/pve/pve-root-ca.pem /etc/pve/pve-root-ca.pem.SAVED
#cp /etc/pve/priv/pve-root-ca.pem /etc/pve/priv/pve-root-ca.pem.SAVED
preferred:
# tar czf ~/SSL_FILE_Backup.tar.gz /etc/pve/pve-root-ca.pem /etc/pve/priv/
note: Please ignore the "Removing leading `/' from member names" message
# tar tzf ~/SSL_FILE_Backup.tar.gz (Trust but verify, always)


That was the hard part, mitigating your potential loss.
Now on to the real work.


1. Obtain your SSL Cert. (PEM and KEY), I used StartSSL(can't beat the value)


2. Place your "Real" Cert files in the : /etc/ssl


Following the normal structure:
PEM FILE:
/etc/ssl/certs/<hostname>.<domain>.<tld>.pem


ex. /etc/ssl/certs/host1.mydomain.net.pem


KEY FILE:
/etc/ssl/private/<hostname>.<domain>.<tld>.key


ex. /etc/ssl/private/host1.mydomain.net.key


INTERMEDIATE FILE:




3. DO NOT FOR ANY REASON REAL OR IMAGINED touch, move, or alter
ANY File in the /etc/pve/ or /etc/pve//priv directories. Yes, it's worth saying twice.
Very Bad things will happen if you do, including the potential for loss of life.


4. Modify the SSLCerticate entries in enabled sites files:


# vi /etc/apache2/sites-enabled/pve.conf /etc/apache2/sites-enabled/pve-redirect.conf


Should look like:


PVE.CONF & PIE-REDIRECT.CONF:


note: First comment out the existing entries, copy them and then modify.
Once again trusting but verifying (my own actions).


# SSLCertificateFile /etc/pve/local/pve-ssl.pem
# SSLCertificateKeyFile /etc/pve/local/pve-ssl.key
# SSLCertificateChainFile /etc/pve/local/sub.class1.server.ca.pem


SSLCertificateFile /etc/ssl/certs/host1.mydomain.netpem
SSLCertificateKeyFile /etc/ssl/private/host1.mydomain.net.key
SSLCertificateChainFile /etc/ssl/certs/sub.class2.server.ca.pem




5. Run the 'pvecm updatecerts --force''


6. Reboot your machine.


7. Check your work accessing the secure URL in your favorite browser:


https://host1.mydomain.net.8006


8. Click on a VT or a VM and click on Console.


All should be working, if not you probably did something bad before you started this process, go back in time and figure out what. Otherwise you may have to reinstall.