I've got three nodes in a cluster and the other two upgraded fine but the third one got this far and then the node showed as down in the UI and I can't SSH in to it either.
Any ideas why this might have happened? I'll have to do a hard reboot to see if it comes up again, otherwise I'll have to connect it to a monitor and keyboard to check it.
ETA: I've tried purging the package with 'dpkg --remove --force-remove-reinstreq proxmox-kernel-7.0.14-19-pve-signed' and clearing the cache with
and doing the upgrade again with 'apt-get dist-upgrade' but it crashes every time, although it does at least recover by itself now without me having to do a hard reboot. There's plenty of free space in /boot, which currently only contains:
I think I've fixed it now. It seems there was a problem with old CPU microcode causing the decompress to crash when using multiple threads. I was able to install the update using single-core execution with 'taskset -c 0 dpkg -i /var/cache/apt/archives/proxmox-kernel-7.0.14-19-pve-signed_7.0.14-19_amd64.deb'.
Then I installed the Intel microcode. Previously dmesg | grep -i microcode showed:
and now it shows:
and testing the multi-thread decompression with 'apt-get reinstall proxmox-kernel-7.0.14-19-pve-signed' and 'ar p /var/cache/apt/archives/proxmox-kernel-7.0.14-19-pve-signed_7.0.14-19_amd64.deb $(ar t /var/cache/apt/archives/proxmox-kernel-7.0.14-19-pve-signed_7.0.14-19_amd64.deb | grep data.tar) | xz -dc -T 0 | tar -tf - > /dev/null' confirms that's working now.
Any ideas why this might have happened? I'll have to do a hard reboot to see if it comes up again, otherwise I'll have to connect it to a monitor and keyboard to check it.
Code:
Starting system upgrade: apt-get dist-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
proxmox-kernel-7.0.14-19-pve-signed
The following packages will be upgraded:
proxmox-kernel-7.0
1 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 131 MB of archives.
After this operation, 1,030 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 proxmox-kernel-7.0.14-19-pve-signed amd64 7.0.14-19 [131 MB]
Get:2 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 proxmox-kernel-7.0 amd64 7.0.14-19 [17.3 kB]
Fetched 131 MB in 1s (107 MB/s)
Selecting previously unselected package proxmox-kernel-7.0.14-19-pve-signed.
(Reading database ... 62080 files and directories currently installed.)
Preparing to unpack .../proxmox-kernel-7.0.14-19-pve-signed_7.0.14-19_amd64.deb ...
Unpacking proxmox-kernel-7.0.14-19-pve-signed (7.0.14-19) ...ETA: I've tried purging the package with 'dpkg --remove --force-remove-reinstreq proxmox-kernel-7.0.14-19-pve-signed' and clearing the cache with
Code:
apt-get clean
apt-get update
dpkg --configure -aand doing the upgrade again with 'apt-get dist-upgrade' but it crashes every time, although it does at least recover by itself now without me having to do a hard reboot. There's plenty of free space in /boot, which currently only contains:
Code:
ls /boot -la
total 101137
drwxr-xr-x 5 root root 4096 Sep 22 13:50 .
drwxr-xr-x 19 root root 4096 Sep 20 15:12 ..
-rw-r--r-- 1 root root 308353 Sep 10 10:16 config-7.0.14-17-pve
drwx------ 3 root root 512 Jan 1 1970 efi
drwxr-xr-x 5 root root 4096 Sep 20 14:54 grub
-rw-r----- 1 root root 76481034 Sep 20 16:16 initrd.img-7.0.14-17-pve
drwxr-xr-x 2 root root 4096 Sep 17 17:13 pve
-rw-r--r-- 1 root root 10478940 Sep 10 10:16 System.map-7.0.14-17-pve
-rw-r--r-- 1 root root 16268392 Sep 10 10:16 vmlinuz-7.0.14-17-pveI think I've fixed it now. It seems there was a problem with old CPU microcode causing the decompress to crash when using multiple threads. I was able to install the update using single-core execution with 'taskset -c 0 dpkg -i /var/cache/apt/archives/proxmox-kernel-7.0.14-19-pve-signed_7.0.14-19_amd64.deb'.
Then I installed the Intel microcode. Previously dmesg | grep -i microcode showed:
Code:
[ 0.000000] x86/CPU: Running old microcode
[ 0.195036] SRBDS: Mitigation: Microcode
[ 0.629323] microcode: Current revision: 0x000000f4and now it shows:
Code:
[ 0.195382] SRBDS: Mitigation: Microcode
[ 0.636117] microcode: Current revision: 0x000000fa
[ 0.636119] microcode: Updated early from: 0x000000f4
Last edited:
