[SOLVED] HTTPS Certificate trouble - tlsv1 alert unknown ca

[udo]

Hi,
we are migrating from our pve3.4-cluster to an new pve4.1 (enterprise repository) cluster.

Do an fresh install on one node (named prox-01) and change the ssl-files like here discribed: http://pve.proxmox.com/wiki/HTTPSCertificateConfiguration
Transfer some VMs - all work well.
Do an fresh install on the second node (prox-02), change the SSL-Files too and join the cluster.
Work, but with an SSL error, because update certs won't work...

But then I got issues with prox-01 (update certs empty the file /etc/pve/nodes/prox-01/pve-ssl.pem).
I replace pve-ssl.pem with the right file (same config like on prox-02).
The Web-SSL config is ok (checked with openssl s_client -connect prox-01...).

But if I logged in on node prox-01 and open an Console - after an short time the Console changed to disconnected, switched back, and so on.

Error messages:

Mar 15 13:36:50 prox-01 pveproxy[3585]: problem with client 172.20.4.200; ssl3_read_bytes: tlsv1 alert unknown ca
Mar 15 13:36:50 prox-01 pveproxy[3585]: proxy detected vanished client connection
Mar 15 13:36:52 prox-01 pveproxy[3585]: problem with client 172.20.4.200; ssl3_read_bytes: tlsv1 alert unknown ca
Mar 15 13:36:52 prox-01 pveproxy[3585]: proxy detected vanished client connection
Mar 15 13:36:55 prox-01 pveproxy[3584]: problem with client 172.20.4.200; ssl3_read_bytes: tlsv1 alert unknown ca
Mar 15 13:36:55 prox-01 pveproxy[3584]: Can't call method "timeout_reset" on an undefined value at /usr/share/perl5/PVE/HTTPServer.pm line 225

The same happens after an reboot of node prox-01.

If I logged in on prox-02, I can open an Console of an prox-01-VM... but get often an message displayed "ssl3_read_bytes: tlsv1 alert unknown ca (596)".

Files looks correct (key is an wildcard-key):

root@prox-01:~# diff /etc/pve/nodes/prox-01/pve-ssl.pem /etc/pve/nodes/prox-02/pve-ssl.pem
root@prox-01:~# diff /etc/pve/nodes/prox-01/pve-ssl.key /etc/pve/nodes/prox-02/pve-ssl.key

pveversion -v
proxmox-ve: 4.1-39 (running kernel: 4.2.8-1-pve)
pve-manager: 4.1-15 (running version: 4.1-15/8cd55b52)
pve-kernel-4.2.6-1-pve: 4.2.6-36
pve-kernel-4.2.8-1-pve: 4.2.8-39
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 1.0-1
pve-cluster: 4.0-33
qemu-server: 4.0-62
pve-firmware: 1.1-7
libpve-common-perl: 4.0-49
libpve-access-control: 4.0-11
libpve-storage-perl: 4.0-42
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.5-9
pve-container: 1.0-46
pve-firewall: 2.0-18
pve-ha-manager: 1.0-24
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u1
lxc-pve: 1.1.5-7
lxcfs: 2.0.0-pve1
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5-pve7~jessie
openvswitch-switch: 2.3.2-2

openssl s_client -CAfile ca.pem -connect prox-01.xxxxxxxxxx.com:8006
CONNECTED(00000003)
depth=2 O = Root CA, OU = http://www.cacert.org, CN = CA Cert Signing Authority, emailAddress = support@cacert.org
verify return:1
depth=1 O = CAcert Inc., OU = http://www.CAcert.org, CN = CAcert Class 3 Root
verify return:1
depth=0 C = DE, ST = Hamburg, L = Hamburg, O = XXXXXXXXXXXX, CN = *.xxxxxxxxxx.com
verify return:1
---
Certificate chain
0 s:/C=DE/ST=Hamburg/L=Hamburg/O=XXXXXXXXXXXX/CN=*.xxxxxxxxxx.com
  i:/O=CAcert Inc./OU=http://www.CAcert.org/CN=CAcert Class 3 Root
1 s:/O=CAcert Inc./OU=http://www.CAcert.org/CN=CAcert Class 3 Root
  i:/O=Root CA/OU=http://www.cacert.org/CN=CA Cert Signing Authority/emailAddress=support@cacert.org
2 s:/O=Root CA/OU=http://www.cacert.org/CN=CA Cert Signing Authority/emailAddress=support@cacert.org
  i:/O=Root CA/OU=http://www.cacert.org/CN=CA Cert Signing Authority/emailAddress=support@cacert.org
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIF1TCCA72gAwIBAgIDAmgIMA0GCSqGSIb3DQEBCwUAMFQxFDASBgNVBAoTC0NB
Y2VydCBJbmMuMR4wHAYDVQQLExVodHRwOi8vd3d3LkNBY2VydC5vcmcxHDAaBgNV
BAMTE0NBY2VydCBDbGFzcyAzIFJvb3QwHhcNMTUwNTE5MTU0NTM4WhcNMTcwNTE4
MTU0NTM4WjB8MQswCQYDVQQGEwJERTEQMA4GA1UECBMHSGFtYnVyZzEQMA4GA1UE
BxMHSGFtYnVyZzEtMCsGA1UEChQkQWxiZXJ0IEJhdWVyIENvbXBhbmllcyBHbWJI
ICYgQ28uIEtHMRowGAYDVQQDFBEqLmFsYmVydGJhdWVyLmNvbTCCAaIwDQYJKoZI
hvcNAQEBBQADggGPADCCAYoCggGBAMjOaoLYRzG2b8Pd+NMJsbQ7E/4cyQk5llGg
ytuvPeiPcQCYx01iKNlAVwLyMZ1VK5CAtkrztReH27R+/2QAJIxA73uAz10jj0XO
QzFu4evLMr1ClHoT4YA1wMUOFSFD6vgCr9eA/zl5Ba96khNzwAmrg846ndnMrLzw
DcSGQG3v+Jdd65DQHu+MNROvLurZ2xEpsLX+C4/jRRGUdWkcOZVXnmtWEZiiftaU
C48//iMJiaTeRsufH1+TG9EfdD7B4IafhWK1+Hsyf+UObw4SBUI7Nv6LnWKJP+gC
Qr/7PvE9Lu/IGpQQrRdwivUqoSCq2UMqK+uqhe3I3/5SjdaaaaluSRtX+mHi2z79
fQjNaRuutRn33hFFV9aYLKnSTf2wY3y3UObe07/F9MFCQX/mBsi5fwiiitMUgAET
8oWEJG3veSgmC0rHRANpoW6cD4YlrVyOQRgGzLuPtFKgc1f3Pn4ysAQ9MO/vxZkA
tKk1L5Wky+NRGEpkcWmC3RuUE2dkQQIDAQABo4IBBjCCAQIwDAYDVR0TAQH/BAIw
ADAOBgNVHQ8BAf8EBAMCA6gwNAYDVR0lBC0wKwYIKwYBBQUHAwIGCCsGAQUFBwMB
BglghkgBhvhCBAEGCisGAQQBgjcKAwMwMwYIKwYBBQUHAQEEJzAlMCMGCCsGAQUF
BzABhhdodHRwOi8vb2NzcC5jYWNlcnQub3JnLzA4BgNVHR8EMTAvMC2gK6Aphido
dHRwOi8vY3JsLmNhY2VydC5vcmcvY2xhc3MzLXJldm9rZS5jcmwwPQYDVR0RBDYw
NIIRKi5hbGJlcnRiYXVlci5jb22gHwYIKwYBBQUHCAWgEwwRKi5hbGJlcnRiYXVl
ci5jb20wDQYJKoZIhvcNAQELBQADggIBAAIlCWYaSbkpJnhXvE9hPsi6uNFfIovv
9d2eiv2MD2A2V3EgqjfMOxWua/LiV6n+zDYzk/R9DYdi1ozwmQ9xnLU8206JMXLJ
PFaqRDtXBqtMckCZuJ2B7yTiPigAPcN9wEOfrpjdKzEx9XjE8bQBx+NmDNgBcwEk
tAlgWichKDHwhwbHWKhgQfRCT7QAnRBTZz1Prmkn04NNK7k9kISRFevOruKFJIBB
d7N09YneYXwfUepmcxmSxXimhUQJC9evV23nnQqUM9dW+H9t4iZ4AgZo/Po2U4IF
kPCE57KzEJcOMNv6EVx6DS5KU0Gmge0L+q1KEjuv9hgHY6yYDqkRnZ/hWOKmSzfc
sFBkPaYFd+ICsJH6P0nZTEPAxKdjvY1FlOfdacuGt2GS9I/bESx9jLPlCkb6FrAV
Q+y4U7EFEPgoBmasMeXGwncXmPtXfeVUkuhwU0Soh4EYLLUGbPe/adOregvAxxQX
iFZ7BwJGmrunDxc31n7+HTf+I6XWnJF2AoVT/ZKqmvSdVp0Wx4tEFiuqlBaBivpV
cvRZjVQlaiN2n0jynODMxNh4W+k81YzIAJgnRZWkmn1tX2Fz71+PXy43QmU//q7R
bSgyRQ/bk7auqlyxlhKemizLluPjjZ5kXalnVA8AMRTZginAwGuz8DTXt6R6gTwv
Wu1Y2Y6AMRHi
-----END CERTIFICATE-----
subject=/C=DE/ST=Hamburg/L=Hamburg/O=XXXXXXXXXXXX/CN=*.xxxxxxxxxx.com
issuer=/O=CAcert Inc./OU=http://www.CAcert.org/CN=CAcert Class 3 Root
---
No client certificate CA names sent
---
SSL handshake has read 6474 bytes and written 607 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-GCM-SHA384
Server public key is 3072 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
  Protocol  : TLSv1.2
  Cipher  : DHE-RSA-AES256-GCM-SHA384
  Session-ID: 6A562C373F9B6BC7548D2C39DAF7CCDDDAB89A4152F308BB073A0D9F6DC7DC26
  Session-ID-ctx:
  Master-Key: E0BEACEB67F4428E5C1DEC279845013A34B8FEBF6B1852A2E9D7AFED6A4DBBD8223EFEF92C45149DF8CB20AD1EFDD5EB
  Key-Arg  : None
  PSK identity: None
  PSK identity hint: None
  SRP username: None
  TLS session ticket lifetime hint: 300 (seconds)
  TLS session ticket:
  0000 - 6f 37 22 95 a0 57 23 82-d9 0e ad 5c e8 59 a7 0c  o7"..W#....\.Y..
  0010 - 97 f7 f5 5e 66 b6 31 f6-1f 3d b4 88 64 ae f0 13  ...^f.1..=..d...
  0020 - 4e e7 3b 67 45 9e c4 5a-fd 75 15 a0 f2 32 36 95  N.;gE..Z.u...26.
  0030 - cf 1a 60 7a 4a ca 17 7a-91 be 58 8a 6d 81 3b 91  ..`zJ..z..X.m.;.
  0040 - d8 9b ec a6 40 16 11 c7-d8 70 32 95 d9 79 73 c6  ....@....p2..ys.
  0050 - 19 81 d4 47 2c 11 00 7e-a4 43 61 11 a4 01 27 ae  ...G,..~.Ca...'.
  0060 - ba b2 4e 28 4c 87 cf 6e-b2 ab 32 40 f6 02 8d 21  ..N(L..n..2@...!
  0070 - f6 39 b9 74 ec d2 92 ab-5c 15 13 9f ed 69 80 de  .9.t....\....i..
  0080 - 1f 17 45 ba a3 2a 73 cc-ec 55 2a dc 17 93 4a f2  ..E..*s..U*...J.
  0090 - 04 33 95 40 d0 d5 e3 a4-56 5c 23 b8 28 24 b3 89  .3.@....V\#.($..

  Start Time: 1458045806
  Timeout  : 300 (sec)
  Verify return code: 0 (ok)
---
read:errno=0

Any hints?

Udo

 

[udo]

Hi,
seems that the issue was the missing content of /etc/pve/nodes/prox-01/pve-ssl.pem and after that the browser cache...

Restart from browser (which I have done before) fix the issue.

sorry for the noise

Udo