I just upgraded to the latest version on my 3 node cluster, and was working on building some Packer scripts to build VM templates. I set up a Proxmox API key, but I couldn't get it to connect due to authentication failures:
Eventually, I started testing the API URL directly, and I started running into errors there too, on both the node directly, and the client:
In the browser, it looks like it's not using SSL at all, as it says "Not Secure" and HTTPS is crossed out, although I can see the self-signed certificate if I click on the certificate viewer.
I tried testing the SSL connection directly:
I get the above error "SSL routines:ssl3_read_n:unexpected eof while reading" on my workstation, as well as on the node itself when testing.
The Proxmox node has:
My workstation has:
I stated looking into that last error, and found a couple of GItHub Issues related to OpenSSL v3.
These are the GitHub issues I found:
https://github.com/openssl/openssl/discussions/24810
https://github.com/curl/curl/issues/7800
I tried setting the ignore option in /etc/ssl/openssl.cnf on the client and server, but it didn't help, even after restarting networking services.
I'm stumped on how to fix this.
Do I need to generate new self-signed certificates?
Is this a problem with the current version of OpenSSL on the server?
Thanks in advance for any advice!
Code:
$ packer build .
proxmox-iso.debian: output will be in this color.
Build 'proxmox-iso.debian' errored after 3 seconds 488 milliseconds: 401 authentication failure
==> Wait completed after 3 seconds 488 milliseconds
==> Some builds didn't complete successfully and had errors:
--> proxmox-iso.debian: 401 authentication failure
==> Builds finished but no artifacts were created.Eventually, I started testing the API URL directly, and I started running into errors there too, on both the node directly, and the client:
Code:
root@pve1:~# curl -v -k -H 'Authorization: packer@pve\!packer=apikeyhere' https://pve1.linuxtek.lan:8006/api2/json/cluster/status
* Trying 172.16.15.15:8006...
* Connected to pve1.linuxtek.lan (172.16.15.15) port 8006 (#0)
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
* subject: OU=PVE Cluster Node; O=Proxmox Virtual Environment; CN=pve1.linuxtek.lan
* start date: Jan 3 06:57:45 2025 GMT
* expire date: Jan 3 06:57:45 2027 GMT
* issuer: CN=Proxmox Virtual Environment; OU=idhere; O=PVE Cluster Manager CA
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* using HTTP/1.x
> GET /api2/json/cluster/status HTTP/1.1
> Host: pve1.linuxtek.lan:8006
> User-Agent: curl/7.88.1
> Accept: */*
> Authorization: packer@pve\!packer=apikeyhere
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/1.1 401 No ticket
< Cache-Control: max-age=0
< Connection: close
< Date: Thu, 13 Mar 2025 13:36:08 GMT
< Pragma: no-cache
< Server: pve-api-daemon/3.0
< Expires: Thu, 13 Mar 2025 13:36:08 GMT
<
* Closing connection 0
* TLSv1.3 (OUT), TLS alert, close notify (256):In the browser, it looks like it's not using SSL at all, as it says "Not Secure" and HTTPS is crossed out, although I can see the self-signed certificate if I click on the certificate viewer.
I tried testing the SSL connection directly:
Code:
root@pve1:~# openssl s_client -connect pve1.linuxtek.lan:8006
CONNECTED(00000003)
depth=0 OU = PVE Cluster Node, O = Proxmox Virtual Environment, CN = pve1.linuxtek.lan
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 OU = PVE Cluster Node, O = Proxmox Virtual Environment, CN = pve1.linuxtek.lan
verify error:num=21:unable to verify the first certificate
verify return:1
depth=0 OU = PVE Cluster Node, O = Proxmox Virtual Environment, CN = pve1.linuxtek.lan
verify return:1
---
Certificate chain
0 s:OU = PVE Cluster Node, O = Proxmox Virtual Environment, CN = pve1.linuxtek.lan
i:CN = Proxmox Virtual Environment, OU = 17ccd30e-1fc9-4d53-be4a-9866f5ad6bdf, O = PVE Cluster Manager CA
a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
v:NotBefore: Jan 3 06:57:45 2025 GMT; NotAfter: Jan 3 06:57:45 2027 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
subject=OU = PVE Cluster Node, O = Proxmox Virtual Environment, CN = pve1.linuxtek.lan
issuer=CN = Proxmox Virtual Environment, OU = ouhere, O = PVE Cluster Manager CA
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1839 bytes and written 403 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 21 (unable to verify the first certificate)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
Protocol : TLSv1.3
Cipher : TLS_AES_256_GCM_SHA384
Session-ID: 51CBDBB6108DFC1E3E93CA664701CCAD550C05813E9C6CBE5C91EEB9ED74AFA0
Session-ID-ctx:
Resumption PSK: AF6A965474E5D69D745BC738780BB4160EE3AF83F6EB6EC28DE01A7157E2740C4BE5A088A763E250B90D4CDAB8502EAF
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 7200 (seconds)
TLS session ticket:
0000 - d6 d9 aa 44 c1 1e 50 f0-1b a5 8c 63 67 5f c4 46 ...D..P....cg_.F
0010 - f9 a0 e7 92 8d 23 18 e7-05 7e 49 9d 90 4f de 51 .....#...~I..O.Q
0020 - ab 4e ca 8d ed 87 45 da-ef 61 75 27 99 c4 34 1c .N....E..au'..4.
0030 - 0e 2f 7e 55 3d fa fc f1-f3 26 48 33 93 e5 a1 5e ./~U=....&H3...^
0040 - 57 76 6a 74 5d 60 fd 62-5d 15 68 72 6c 45 5c 0e Wvjt]`.b].hrlE\.
0050 - 12 ce 64 62 0e db 1f c0-79 95 13 f9 e6 f9 06 1d ..db....y.......
0060 - c0 34 11 5d 2b 38 73 6c-97 6b ff c5 08 f8 74 74 .4.]+8sl.k....tt
0070 - 8d 32 bf 8b d8 d7 7e 2b-60 8e b5 4c 6c f2 2d 47 .2....~+`..Ll.-G
0080 - 90 2a 90 ef ab 2f 87 d0-06 db 00 71 77 2e bc a6 .*.../.....qw...
0090 - 28 d2 21 56 e5 d1 9c aa-b5 1a 69 e7 70 c1 51 34 (.!V......i.p.Q4
00a0 - 42 b1 0b 57 36 5c 2b 16-f2 fe 02 21 1c 95 80 cb B..W6\+....!....
00b0 - c2 6d 48 74 a4 d1 9b 98-26 d7 d4 26 8d ac 0e 99 .mHt....&..&....
00c0 - 46 7d 25 44 df f7 cc 2b-e2 ae 5c c2 e1 a6 77 9c F}%D...+..\...w.
Start Time: 1741873200
Timeout : 7200 (sec)
Verify return code: 21 (unable to verify the first certificate)
Extended master secret: no
Max Early Data: 0
---
read R BLOCK
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
Protocol : TLSv1.3
Cipher : TLS_AES_256_GCM_SHA384
Session-ID: DB97D80F6B074EA52B7B66C37B8DC61366D06A4BAB983D8BB405D9E2FF26EBBD
Session-ID-ctx:
Resumption PSK: 192E198306D143E91CFA690ECDD52E9C91A8668C1A57B95719B0FE59CFAADDDFF6F2AC0DCC5CCB6E74A62CC0F7E8414E
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 7200 (seconds)
TLS session ticket:
0000 - d6 d9 aa 44 c1 1e 50 f0-1b a5 8c 63 67 5f c4 46 ...D..P....cg_.F
0010 - de ed 79 60 c6 bf 7a e7-02 af 52 c6 1d 58 1d 26 ..y`..z...R..X.&
0020 - bd bc ef 44 d4 3a de a9-4b a3 4f 9c 97 97 f8 34 ...D.:..K.O....4
0030 - 44 98 78 c2 3e 74 dc 25-e1 8d b2 33 17 bc b0 f3 D.x.>t.%...3....
0040 - 2a bf 19 ac 47 ab a0 db-cd e4 9b ff 17 a2 eb 88 *...G...........
0050 - fa d1 64 5c e4 f3 14 fd-c8 57 60 a8 1f 82 ca 7e ..d\.....W`....~
0060 - 8c bc 64 2c 33 b2 97 08-11 f6 16 9a 09 5f e3 7f ..d,3........_..
0070 - 24 52 15 5a 2f 37 9e 86-0a 30 d5 83 bc f1 7a 44 $R.Z/7...0....zD
0080 - 5f 74 fa cd 65 66 86 fa-97 87 3b 72 45 4c 9f d9 _t..ef....;rEL..
0090 - 4e 66 e8 03 e1 cd f5 9b-ee c5 af f9 c7 92 52 c5 Nf............R.
00a0 - df 85 74 f8 c1 74 ba 2c-c0 a1 31 ff 77 ef 4a 82 ..t..t.,..1.w.J.
00b0 - 46 11 99 07 ff 38 fd b5-3d a3 97 11 37 c4 ee 44 F....8..=...7..D
00c0 - 7a 52 0c a9 80 72 5e e7-32 f3 18 f9 98 22 d3 07 zR...r^.2...."..
Start Time: 1741873200
Timeout : 7200 (sec)
Verify return code: 21 (unable to verify the first certificate)
Extended master secret: no
Max Early Data: 0
---
read R BLOCK
806B989E107B0000:error:0A000126:SSL routines:ssl3_read_n:unexpected eof while reading:../ssl/record/rec_layer_s3.c:322:I get the above error "SSL routines:ssl3_read_n:unexpected eof while reading" on my workstation, as well as on the node itself when testing.
The Proxmox node has:
Code:
openssl/stable,now 3.0.15-1~deb12u1 amd64 [installed]My workstation has:
Code:
openssl/jammy-updates,jammy-security,now 3.0.2-0ubuntu1.19 amd64 [installed]I stated looking into that last error, and found a couple of GItHub Issues related to OpenSSL v3.
These are the GitHub issues I found:
https://github.com/openssl/openssl/discussions/24810
https://github.com/curl/curl/issues/7800
I tried setting the ignore option in /etc/ssl/openssl.cnf on the client and server, but it didn't help, even after restarting networking services.
Code:
[ssl_default_sect]
Options = IgnoreUnexpectedEOF
or
SSL_OP_IGNORE_UNEXPECTED_EOFI'm stumped on how to fix this.
Do I need to generate new self-signed certificates?
Is this a problem with the current version of OpenSSL on the server?
Thanks in advance for any advice!
