"invalid: subscription information too old" and "Connection error 596: Broken pipe"

udotirol

Well-Known Member
Mar 9, 2018
61
20
48
52
Hi,

one of our nodes feels unhappy about its subscription state and displays "invalid: subscription information too old" in the node's subscription page:

sub1.png

When I click "Check", a popup opens and displays "Connection error 596: Broken pipe".

sub2.png

Nothing conclusive in the logs, maybe anyone else has an idea what to do about it?

Thanks

Udo
 
On a hunch - check if the system tries to connect to shop.proxmox.com via ipv6, but cannot (there was a change in the subscription check code quite a while ago, where broken dual-stack setups could not check their subscription anymore)

else to debug this:
* check on the commandline what `pvesubscription get` and `pvesubscription update` say
* see if you can reach shop.proxmox.com with curl (on port 443)

I hope this helps!
 
thanks.

I can ping & connect to shop.proxmox.com on the node's commandline, but "pvesubscription update" throws an ugly exception:

Code:
root@sisyphos:~# RUST_BACKTRACE=full pvesubscription update
thread '<unnamed>' panicked at 'Failed to add native certificate too root store: UnsupportedCriticalExtension', /usr/share/cargo/registry/ureq-2.4.0/src/rtls.rs:73:14
stack backtrace:
   0:     0x7f1773785c3c - <unknown>
   1:     0x7f17737a161e - <unknown>
   2:     0x7f1773766191 - <unknown>
   3:     0x7f1773771fe5 - <unknown>
   4:     0x7f1773771c42 - <unknown>
   5:     0x7f1773772501 - <unknown>
   6:     0x7f1773785f77 - <unknown>
   7:     0x7f1773785d54 - <unknown>
   8:     0x7f17737721b2 - <unknown>
   9:     0x7f17733ded23 - <unknown>
  10:     0x7f17733dee63 - <unknown>
  11:     0x7f17735b4542 - <unknown>
  12:     0x7f177358e2cb - <unknown>
  13:     0x7f1773757d81 - <unknown>
  14:     0x7f17733d4d22 - <unknown>
  15:     0x7f177358d368 - <unknown>
  16:     0x7f1773580752 - <unknown>
  17:     0x7f177358149d - <unknown>
  18:     0x7f177346ce9f - <unknown>
  19:     0x7f1773443c42 - <unknown>
  20:     0x7f17734446de - <unknown>
  21:     0x7f1773443ca6 - <unknown>
  22:     0x55c0243e2157 - Perl_pp_entersub
  23:     0x55c0243d8846 - Perl_runops_standard
  24:     0x55c024346b1c - perl_run
  25:     0x55c024319482 - main
  26:     0x7f17746ded0a - __libc_start_main
  27:     0x55c0243194ca - _start
  28:                0x0 - <unknown>
fatal runtime error: failed to initiate panic, error 5
Aborted

Previously, the node was running PVE 7.3-4, but the error persists even after upgrading to the latest 7.4-3
 
Last edited:
hmm - again more on a hunch - do you have any local CA-certificates in the system? (/usr/local/share/ca-certificates) - and if yes - could it be that they contain something which is not really well-formed for a certificate?
 
yes, there are some local CA certificates in that directory. They have been put there in Nov 2019, so that's quite a while :)

I however noticed that one of the certificates didn't have .crt but .pem as a filename extension. After listing all known certificates from /etc/ssl/certs/ca-certificates.crt I noticed, that this one certificate was missing from the system.

After renaming the .pem to .crt and running update-ca-certificates, the certificate got successfully imported, but that didn't impress pvesubscription update:

Code:
$ update-ca-certificates -v --fresh
Clearing symlinks in /etc/ssl/certs...
done.
Updating certificates in /etc/ssl/certs...
Doing .
link D-TRUST_Root_Class_3_CA_2_EV_2009.pem -> d4dae3dd.0
link Entrust_Root_Certification_Authority_-_G4.pem -> 5e98733a.0
link Amazon_Root_CA_1.pem -> ce5e74ef.0
link Autoridad_de_Certificacion_Firmaprofesional_CIF_A62634068.pem -> 3bde41ac.0
link Certum_Trusted_Network_CA_2.pem -> 40193066.0
[...]
link ExampleCom_CA_HTTPS_2018.pem -> 7fdfe7ce.0
link ExampleCom_CA_2018.pem -> ffa5834b.0
[...]
link IdenTrust_Public_Sector_Root_CA_1.pem -> 1e08bfd1.0
link Trustwave_Global_Certification_Authority.pem -> f249de83.0
link certSIGN_ROOT_CA.pem -> 8d86cdd1.0
131 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
$
$ RUST_BACKTRACE=full pvesubscription update
thread '<unnamed>' panicked at 'Failed to add native certificate too root store: UnsupportedCriticalExtension', /usr/share/cargo/registry/ureq-2.4.0/src/rtls.rs:73:14
stack backtrace:
   0:     0x7f9c8bcb92e0 - <unknown>
   1:     0x7f9c8bcd671e - <unknown>
   2:     0x7f9c8bca46f5 - <unknown>
   3:     0x7f9c8bca6eb4 - <unknown>
   4:     0x7f9c8bca6b01 - <unknown>
   5:     0x7f9c8bca748a - <unknown>
   6:     0x7f9c8bcb9637 - <unknown>
   7:     0x7f9c8bcb942c - <unknown>
   8:     0x7f9c8bca7182 - <unknown>
   9:     0x7f9c8b900693 - <unknown>
  10:     0x7f9c8b9007c3 - <unknown>
  11:     0x7f9c8badda27 - <unknown>
  12:     0x7f9c8bac857b - <unknown>
  13:     0x7f9c8bc8b5bf - <unknown>
  14:     0x7f9c8b8f5ad2 - <unknown>
  15:     0x7f9c8bac8898 - <unknown>
  16:     0x7f9c8baaedd4 - <unknown>
  17:     0x7f9c8baaf9dd - <unknown>
  18:     0x7f9c8b96b6eb - <unknown>
  19:     0x7f9c8b9674de - <unknown>
  20:     0x7f9c8b967fb7 - <unknown>
  21:     0x7f9c8b967546 - <unknown>
  22:     0x558e06d62157 - Perl_pp_entersub
  23:     0x558e06d58846 - Perl_runops_standard
  24:     0x558e06cc6b1c - perl_run
  25:     0x558e06c99482 - main
  26:     0x7f9c8cc16d0a - __libc_start_main
  27:     0x558e06c994ca - _start
  28:                0x0 - <unknown>
fatal runtime error: failed to initiate panic, error 5
Aborted
 
Last edited:
Allright, after temporarily removing the local CA certificates from /usr/local/share/ca-certificates and running "update-ca-certificates --fresh" again, pvesubscription update successfully completes.

So the issue must have something to do with those custom CA certificates, but as they have been in use since 2018 on a very huge number of devices and operating systems, I have my doubts that the problem is really the certificates themselves.

The exception reads "UnsupportedCriticalExtension". When investigating the certificates, they do have two defined critical extensions:

Code:
$ openssl x509 -noout -text < ExampleCom_CA_2018.crt
[...]
X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:TRUE
[...]
            X509v3 CRL Distribution Points: critical

                Full Name:
                  URI:https://foo.example.com/CA/foo.CRL
[...]
$
$ openssl x509 -noout -text < ExampleCom_CA_HTTPS_2018.crt
[...]
X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:TRUE
[...]
$

So my best guess is that some tool proxmox uses to interact with certificates (ureq?) doesn't like one of the "critical" X509 extensions, but I am a little bit lost here as how to proceed.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!