VNC TLS handshake failed

qk4l

New Member
Oct 24, 2010
15
0
1
St.Petersburg
Hi, I`m testing Proxmox 2.
After successful creating and install one VM, I start create another but get error in console tab.

Error: TLS handshake failed javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateExceprion: certificate does not match​

And all new VM get the same error.

I don`t configure any other parameters expect mod-ssl`s files for apache2.

SSLCertificateFile /etc/pve/local/pve-ssl.pem​
SSLCertificateKeyFile /etc/pve/local/pve-ssl.key​


And whats more strange first VM in this cause work correctly.

 
Same problem on a new host, only when viewing a KVM container.

I realised that the host was also missing the
/etc/pve/local/pve-ssl.pem
/etc/pve/local/pve-ssl.key

When i installed it. So I have coppied it from one machine to an other so apache can run.

Did you have a fix?
 
Last edited:
I don`t configure any other parameters expect mod-ssl`s files for apache2.
SSLCertificateFile /etc/pve/local/pve-ssl.pem​
SSLCertificateKeyFile /etc/pve/local/pve-ssl.key​

Confused - what did you change exactly? It worked before you changed that?
 
I've created a new certificate. Since then, I get the following error.

Error: TLS handshake failed javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateExceprion: certificate does not match
 
I have copied the certs from one node to an other, so they must be from the same ca-root
 
Ok maybe a dump question, but how exactly doe you generate the certs again?
 
Thanks that worked.

I had tryed that before but then it didn't worked. So I have deleted the wrong certs en then runned it again and then it worked. So first remove the certs file.
 
Hi folks,

I have a similar issue but I didn't get what exactly needs to be done to resolve it.

Here is what I did:

1. Create private key and cert request for *.domain.tld (wildcard certificate)
2. Request new certificate for *.domain.tld from StartSSL
3. Copied cert key to /etc/pve/local/pve-ssl.key.
4. Copied cert pem to /etc/pve/local/pve-ssl.pem
5. Copied StartSSL public root certs (1st and 2nd tier in combined file) to /etc/pve/pve-root-ca.pem
6. Restart Apache2 via /etc/init.d/apache2 restart

After this the Website on :8006 is signed correctly. However when I am opening the console via java applet I still get the names TLS handshake error mentioned above. Also quitting the browser and flushing it's cache did not help.

Also "pvecm updatecerts" did not help. :-(

Any hints? What needs to be done to get an own certificate running? Or is it that I can't use a wildcard cert here (limitation of java?)?

Thanks :)


--------
EDIT:

In reference to my posting here:

According to your(dietmar's) statement it is correct that the self-signed certificate only includes the hostname, not the FQDN (which I find to be quite unusual by the way). I assume the java applet itself is also somehow signed during installation or at least there is a parameter set anywhere which also only includes the hostname, not the complete FQDN. As long as the SSL certificate's common name fits to that java applets setting the VNC console is working just fine. But as soon as one changes the SSL certificates there seems to be a gab between them and the java applet which results in the error message "Error: TLS handshake failed javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateExceprion: certificate does not match"

This is only an assumption from my side but sound quite reasonable to me.
 
Last edited:
Ok I managed to replace the SSL private and public keys in /etc/pve/local and I'm getting the same error from the Java applet. I need some explanation about:

/etc/pve/authkey.pub
/etc/pve/pve-root-ca.pem <--- this should be the CA root public certificate (?)
/etc/pve/pve-www.key
/etc/pve/priv/pve-root-ca.key

What are these for? What do I need to replace to make VNC work? Keep in mind that many of us does use certificates of 3rd-party authorities so we only have the machine private key, the CSR and the certificate itself.

Thank you,
Alessandro.
 
What are these for? What do I need to replace to make VNC work?

First, VNC works out of the box. There should be no need to modify the certificates.

Keep in mind that many of us does use certificates of 3rd-party authorities so we only have the machine private key, the CSR and the certificate itself.

The idea is that 'pve-root-ca.pem' is a CA witch signs all node member certificates. That way all nodes in the cluster are trusted members.

So if you want to replace certificates you should probably replace 'pve-root-ca.pem' (yes, this need to be a CA)
 
The idea is that 'pve-root-ca.pem' is a CA witch signs all node member certificates. That way all nodes in the cluster are trusted members.

So if you want to replace certificates you should probably replace 'pve-root-ca.pem' (yes, this need to be a CA)
I did that: the first certificate the Java applet asks me to accept is the right one, the second one isn't (is signed as "www.proxmox.com"). I tried replacing pve-root-ca.pem with the certification chain and with the root CA but nothing works. Some hints would be appreciated. ;)
 
Last edited by a moderator:
6. Restart Apache2 via /etc/init.d/apache2 restart

After this the Website on :8006 is signed correctly. However when I am opening the console via java applet I still get the names TLS handshake error mentioned above. Also quitting the browser and flushing it's cache did not help.

Does it help if you restart the node (to make sure all services read the new certificate)?
 
I had the same thing with beta3, clicking 'reload' in the VNC window finally fixed it somehow. I also tried clearing the browser cache etc before that.

/K
 
Did anyone ever get this to work in a predictable manner? I have been trying to change my certificate for the last 3 days.
I started by following the procedure laid out on http://pve.proxmox.com/wiki/HTTPSCertificateConfiguration . Including a reboot.
That didn't work. I am using a StartSSL like loredo(above). I even removed the sub.class1.server.ca.pem from the ca.crt file.

The ProxMox GUI and Apache still worked fine, however still no Java console. So I did "pvecm updatecerts" and rebooted.
I then removed the " /etc/pve/pve-root-ca.pem" file and ran "pvecm updatecerts --force" and rebooted. I was backed to a
self-signed cert and Java console worked just fine. That's not optimal. So I restarted and I once again have the PM GUI working.

I would really like to get some insight on what needs to be done to isolate where the Java Console is getting it's certificate information from. As pointed out in other threads it's using www.proxmox.com by default.
It seems that some where in the process of "pvecm updatecerts --force" it is not reading my CA cert or just building it's own.

If anyone can shed some insight on what I need to do to address the comment at the bottom of the aforementioned Wiki instructions
"It is important to change /etc/pve/pve-www.pem and /etc/pve/pve-root-ca.pem because otherwise VM console won't load due to a Java cert validation error"

Specifically what content is expected in those files from a third-party CA ?
 
hello,

01. Request new certificate for *.domain.com from StartSSL
02. Edit /etc/apache2/sites-enabled/pve.conf
...
#SSLCertificateFile /etc/pve/local/pve-ssl.pem
#SSLCertificateKeyFile /etc/pve/local/pve-ssl.key


SSLCertificateFile /etc/pve/local/domain.com_certificate.crt
SSLCertificateKeyFile /etc/pve/local/domain.com_private_key.key

SSLCertificateChainFile /etc/pve/local/sub.class2.server.ca.pem
SSLCACertificateFile /etc/pve/local/ca.pem
SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown
CustomLog /var/log/apache2/ssl_request_log "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
...
03. Restart Apache /etc/init.d/apache2 restart
 
Thanks woodgee for the response. I have the items you pointed out in place and verified.
They seem to only address the Apache SSL component. So my SSL to port 8006 is working perfectly.
It's the TLS error with Java that's the root of my current issue. I can't access a console session.
After lots of investigating, I'm guessing this is an issue because I don't have access to the key repository that the
Java JAR file is using. I don't know where the repository is. If I did I could update the repository with the 'keytool' utility.
I haven't figured that part out yet. I do know where the CA repository is but not the actual VNC app repository.
I suspect that for security reasons that update is not possible, because I would also need the repository password.
At this point I am just going to pursue finding out how to get the 'pvecm updatecerts' to look at the newly created cert.
It seems that when you do the update it's not working. So I'm guessing the filenames or locations are hard coded.
If you do a 'pvecm updatecerts --force' it simply recreates the default certs.

The default Java console certs are for www.proxmox.com and the issue is they don't match my StartSSL certs.
So Java and the browser are rightfully flagging them as not valid. Just need to figure out how to get the update
done so they match.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!