Problem after updating kernel

T

tlex

Member
Mar 9, 2021
103
14
23
43
So I have to admit up front.. I'm a newbie with linux...
I tried to update my kernel with apt update && apt install pve-kernel-5.15

Everything went fine and I rebooted but now when I try to update I always get that issue :
Code: 
apt-get dist-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
4 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Setting up pve-kernel-5.13.19-3-pve (5.13.19-6) ...
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 5.13.19-3-pve /boot/vmlinuz-5.13.19-3-pve
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 5.13.19-3-pve /boot/vmlinuz-5.13.19-3-pve
update-initramfs: Generating /boot/initrd.img-5.13.19-3-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/C6D5-A6CE
    Copying kernel and creating boot-entry for 5.13.19-3-pve
    Copying kernel and creating boot-entry for 5.15.12-1-pve
    Copying kernel and creating boot-entry for 5.15.7-1-pve
Copying and configuring kernels on /dev/disk/by-uuid/C6D5-D464
    Copying kernel and creating boot-entry for 5.13.19-3-pve
    Copying kernel and creating boot-entry for 5.15.12-1-pve
    Copying kernel and creating boot-entry for 5.15.7-1-pve
run-parts: executing /etc/kernel/postinst.d/kdump-tools 5.13.19-3-pve /boot/vmlinuz-5.13.19-3-pve
kdump-tools: Generating /var/lib/kdump/initrd.img-5.13.19-3-pve
mkinitramfs: failed to determine device for /
mkinitramfs: workaround is MODULES=most, check:
grep -r MODULES /var/lib/kdump/initramfs-tools

Error please report bug on initramfs-tools
Include the output of 'mount' and 'cat /proc/mounts'
update-initramfs: failed for /var/lib/kdump/initrd.img-5.13.19-3-pve with 1.
run-parts: /etc/kernel/postinst.d/kdump-tools exited with return code 1
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/pve-kernel-5.13.19-3-pve.postinst line 19.
dpkg: error processing package pve-kernel-5.13.19-3-pve (--configure):
 installed pve-kernel-5.13.19-3-pve package post-installation script subprocess returned error exit status 2
Setting up pve-kernel-5.15.12-1-pve (5.15.12-2) ...
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 5.15.12-1-pve /boot/vmlinuz-5.15.12-1-pve
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 5.15.12-1-pve /boot/vmlinuz-5.15.12-1-pve
update-initramfs: Generating /boot/initrd.img-5.15.12-1-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/C6D5-A6CE
    Copying kernel and creating boot-entry for 5.13.19-3-pve
    Copying kernel and creating boot-entry for 5.15.12-1-pve
    Copying kernel and creating boot-entry for 5.15.7-1-pve
Copying and configuring kernels on /dev/disk/by-uuid/C6D5-D464
    Copying kernel and creating boot-entry for 5.13.19-3-pve
    Copying kernel and creating boot-entry for 5.15.12-1-pve
    Copying kernel and creating boot-entry for 5.15.7-1-pve
run-parts: executing /etc/kernel/postinst.d/kdump-tools 5.15.12-1-pve /boot/vmlinuz-5.15.12-1-pve
kdump-tools: Generating /var/lib/kdump/initrd.img-5.15.12-1-pve
mkinitramfs: failed to determine device for /
mkinitramfs: workaround is MODULES=most, check:
grep -r MODULES /var/lib/kdump/initramfs-tools

Error please report bug on initramfs-tools
Include the output of 'mount' and 'cat /proc/mounts'
update-initramfs: failed for /var/lib/kdump/initrd.img-5.15.12-1-pve with 1.
run-parts: /etc/kernel/postinst.d/kdump-tools exited with return code 1
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/pve-kernel-5.15.12-1-pve.postinst line 19.
dpkg: error processing package pve-kernel-5.15.12-1-pve (--configure):
 installed pve-kernel-5.15.12-1-pve package post-installation script subprocess returned error exit status 2
dpkg: dependency problems prevent configuration of pve-kernel-5.13:
 pve-kernel-5.13 depends on pve-kernel-5.13.19-3-pve; however:
  Package pve-kernel-5.13.19-3-pve is not configured yet.

dpkg: error processing package pve-kernel-5.13 (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of pve-kernel-5.15:
 pve-kernel-5.15 depends on pve-kernel-5.15.12-1-pve; however:
  Package pve-kernel-5.15.12-1-pve is not configured yet.

dpkg: error processing package pve-kernel-5.15 (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 pve-kernel-5.13.19-3-pve
 pve-kernel-5.15.12-1-pve
 pve-kernel-5.13
 pve-kernel-5.15
E: Sub-process /usr/bin/dpkg returned an error code (1)

I tried the following commands with the same results :
Code: 
dpkg --configure -a

apt-get autoclean

apt --fix-broken install

Can anyone help me with that ?
 
These 2 packages seam to be linked to my problem but I don't know what to do :(

Code: 
dpkg -C
The following packages have been unpacked but not yet configured.
They must be configured using dpkg --configure or the configure
menu option in dselect for them to work:
 pve-kernel-5.13      Latest Proxmox VE Kernel Image
 pve-kernel-5.15      Latest Proxmox VE Kernel Image

The following packages are only half configured, probably due to problems
configuring them the first time.  The configuration should be retried using
dpkg --configure <package> or the configure menu option in dselect:
 pve-kernel-5.13.19-3-pve The Proxmox PVE Kernel Image
 pve-kernel-5.15.12-1-pve The Proxmox PVE Kernel Image
 
Can you please attach /var/log/apt/history.log here /var/log/apt/term.log? So that we have a better picture of what happened on that host package-management wise.

The following command output would be good to have too:
Bash: 
pveversion -v
lsblk
cat /proc/cmdline
 
did you try doing what the error message tells you? you have kdump-tools installed, that wants to generate the kdump initrd, but can't determine the FS or / (is it on ZFS?). it suggest configuring the initrd generation to just copy most kernel modules instead of attempting auto-generation:

Code: 
kdump-tools: Generating /var/lib/kdump/initrd.img-5.15.12-1-pve
mkinitramfs: failed to determine device for /
mkinitramfs: workaround is MODULES=most, check:
grep -r MODULES /var/lib/kdump/initramfs-tools
 
  • Like
Reactions: oguz
fabian said:
did you try doing what the error message tells you? you have kdump-tools installed, that wants to generate the kdump initrd, but can't determine the FS or / (is it on ZFS?). it suggest configuring the initrd generation to just copy most kernel modules instead of attempting auto-generation:

Code: 
kdump-tools: Generating /var/lib/kdump/initrd.img-5.15.12-1-pve
mkinitramfs: failed to determine device for /
mkinitramfs: workaround is MODULES=most, check:
grep -r MODULES /var/lib/kdump/initramfs-tools
Click to expand...
I wish I knew how to do that..
grep - r MODULES returns :
Code: 
grep -r MODULES /var/lib/kdump/initramfs-tools
/var/lib/kdump/initramfs-tools/initramfs.conf:# MODULES: [ most | netboot | dep | list ]
/var/lib/kdump/initramfs-tools/initramfs.conf:MODULES=dep
 
t.lamprecht said:
Can you please attach /var/log/apt/history.log here /var/log/apt/term.log? So that we have a better picture of what happened on that host package-management wise.

The following command output would be good to have too:
Bash: 
pveversion -v
lsblk
cat /proc/cmdline
Click to expand...
Thanks, I really appreciate your help.
As you will see in the logs I tried multiple things / times to fix this since upgrading the kernel but since I'm far from being an expert at this I hope I didn't break too many things...
Initially I tried to upgrade the kernel because I had random server crash (ie 1 every 2-3 days) and that is also why I've installed kdump.
So far, except for that error that I now have with the package manager I have to say that the server didn't crash yet.

pveversion -v:
Code: 
proxmox-ve: 7.1-1 (running kernel: 5.15.12-1-pve)
pve-manager: 7.1-10 (running version: 7.1-10/6ddebafe)
pve-kernel-helper: 7.1-8
pve-kernel-5.15.7-1-pve: 5.15.7-1
pve-kernel-5.13.19-2-pve: 5.13.19-4
ceph-fuse: 15.2.15-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.1
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-5
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.1-2
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.1-1
libpve-storage-perl: 7.0-15
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.3.0-1
proxmox-backup-client: 2.1.3-1
proxmox-backup-file-restore: 2.1.3-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-5
pve-cluster: 7.1-3
pve-container: 4.1-3
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-4
pve-ha-manager: 3.3-1
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.0-3
pve-xtermjs: 4.12.0-1
qemu-server: 7.1-4
smartmontools: 7.2-pve2
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.2-pve1

lsblk
Code: 
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    0   1.6T  0 disk
├─sda1        8:1    0   1.6T  0 part
└─sda9        8:9    0     8M  0 part
sdb           8:16   0   1.6T  0 disk
├─sdb1        8:17   0   1.6T  0 part
└─sdb9        8:25   0     8M  0 part
zd0         230:0    0    32G  0 disk
├─zd0p1     230:1    0    31G  0 part
├─zd0p2     230:2    0     1K  0 part
└─zd0p5     230:5    0   975M  0 part
zd16        230:16   0    32G  0 disk
├─zd16p1    230:17   0   512M  0 part
├─zd16p2    230:18   0  30.5G  0 part
└─zd16p3    230:19   0   976M  0 part
zd32        230:32   0    60G  0 disk
├─zd32p1    230:33   0   100M  0 part
├─zd32p2    230:34   0    16M  0 part
└─zd32p3    230:35   0  59.9G  0 part
zd48        230:48   0    32G  0 disk
├─zd48p1    230:49   0   260M  0 part
└─zd48p2    230:50   0  31.7G  0 part
zd64        230:64   0    40G  0 disk
├─zd64p1    230:65   0   100M  0 part
├─zd64p2    230:66   0    16M  0 part
├─zd64p3    230:67   0  39.4G  0 part
└─zd64p4    230:68   0   507M  0 part
zd80        230:80   0     1M  0 disk
zd96        230:96   0    40G  0 disk
├─zd96p1    230:97   0   100M  0 part
├─zd96p2    230:98   0    16M  0 part
├─zd96p3    230:99   0  39.4G  0 part
└─zd96p4    230:100  0   507M  0 part
zd112       230:112  0    32G  0 disk
├─zd112p1   230:113  0    32G  0 part
├─zd112p5   230:117  0    30G  0 part
└─zd112p6   230:118  0   1.6G  0 part
zd128       230:128  0    10G  0 disk
├─zd128p1   230:129  0  1007K  0 part
├─zd128p2   230:130  0   512M  0 part
└─zd128p3   230:131  0   9.5G  0 part
nvme1n1     259:0    0 931.5G  0 disk
├─nvme1n1p1 259:1    0  1007K  0 part
├─nvme1n1p2 259:2    0   512M  0 part
└─nvme1n1p3 259:3    0 899.5G  0 part
nvme0n1     259:4    0 931.5G  0 disk
├─nvme0n1p1 259:5    0  1007K  0 part
├─nvme0n1p2 259:6    0   512M  0 part
└─nvme0n1p3 259:7    0 899.5G  0 part

cat /proc/cmdline
Code: 
initrd=\EFI\proxmox\5.15.12-1-pve\initrd.img-5.15.12-1-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs nmi_watchdog=1 crashkernel=0M-2G:128M,2G-6G:256M,6G-8G:512M,8G-:768M
 

Attachments

tlex said:
I wish I knew how to do that..
grep - r MODULES returns :
Code: 
grep -r MODULES /var/lib/kdump/initramfs-tools
/var/lib/kdump/initramfs-tools/initramfs.conf:# MODULES: [ most | netboot | dep | list ]
/var/lib/kdump/initramfs-tools/initramfs.conf:MODULES=dep
Click to expand...
ok so I tried changing MODULES to most in /var/lib/kdump/initramfs-tools/initramfs.conf
but I don't know what to do next..
 
IIRC, the file /var/lib/kdump/initramfs-tools/initramfs.conf is generated, so changing MODULES there does not help.

The ZFS root + kdump combination unfortunately doesn't work currently, although a patch does exists in a related bug report.
 
I think simply commenting out the sed .. line that overrides MODULES in the kdump initramfs config is enough (line 52 in /etc/kernel/postinst.d/kdump-tools) - the kdump initramfs will then be bigger and use the same modules as the regular one, with no chance of breaking the normal initrd generation.. or removing/puring kdump-tools if you don't use it ;)
 
  • Like
Reactions: Okeur and fiona
fabian said:
I think simply commenting out the sed .. line that overrides MODULES in the kdump initramfs config is enough (line 52 in /etc/kernel/postinst.d/kdump-tools) - the kdump initramfs will then be bigger and use the same modules as the regular one, with no chance of breaking the normal initrd generation.. or removing/puring kdump-tools if you don't use it ;)
Click to expand...
Thank you very much Fabian, all good now :)
Really appreciated !
 
Hello,

Just to mention the solution provided by Fabian worked also in my case.
I was unable to upgrade my proxmox due to kdump, and unable to uninstall kdump either (error below).

So thanks Fabian !

Code: 
root@pve:/var/lib/kdump/initramfs-tools# apt remove --purge kdump-tools
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be REMOVED:
  kdump-tools*
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
4 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n]
Setting up pve-kernel-5.4.174-2-pve (5.4.174-2) ...
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 5.4.174-2-pve /boot/vmlinuz-5.4.174-2-pve
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 5.4.174-2-pve /boot/vmlinuz-5.4.174-2-pve
update-initramfs: Generating /boot/initrd.img-5.4.174-2-pve
run-parts: executing /etc/kernel/postinst.d/kdump-tools 5.4.174-2-pve /boot/vmlinuz-5.4.174-2-pve
kdump-tools: Generating /var/lib/kdump/initrd.img-5.4.174-2-pve
mkinitramfs: failed to determine device for /
mkinitramfs: workaround is MODULES=most, check:
grep -r MODULES /var/lib/kdump/initramfs-tools

Error please report bug on initramfs-tools
Include the output of 'mount' and 'cat /proc/mounts'
update-initramfs: failed for  with 1.
run-parts: /etc/kernel/postinst.d/kdump-tools exited with return code 1
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/pve-kernel-5.4.174-2-pve.postinst line 19.
dpkg: error processing package pve-kernel-5.4.174-2-pve (--configure):
 installed pve-kernel-5.4.174-2-pve package post-installation script subprocess returned error exit status 2
dpkg: dependency problems prevent configuration of pve-kernel-5.4:
 pve-kernel-5.4 depends on pve-kernel-5.4.174-2-pve; however:
  Package pve-kernel-5.4.174-2-pve is not configured yet.

dpkg: error processing package pve-kernel-5.4 (--configure):
 dependency problems - leaving unconfigured
Setting up pve-kernel-5.4.128-1-pve (5.4.128-2) ...
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 5.4.128-1-pve /boot/vmlinuz-5.4.128-1-pve
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 5.4.128-1-pve /boot/vmlinuz-5.4.128-1-pve
update-initramfs: Generating /boot/initrd.img-5.4.128-1-pve
run-parts: executing /etc/kernel/postinst.d/kdump-tools 5.4.128-1-pve /boot/vmlinuz-5.4.128-1-pve
kdump-tools: Generating /var/lib/kdump/initrd.img-5.4.128-1-pve
mkinitramfs: failed to determine device for /
mkinitramfs: workaround is MODULES=most, check:
grep -r MODULES /var/lib/kdump/initramfs-tools

Error please report bug on initramfs-tools
Include the output of 'mount' and 'cat /proc/mounts'
update-initramfs: failed for  with 1.
run-parts: /etc/kernel/postinst.d/kdump-tools exited with return code 1
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/pve-kernel-5.4.128-1-pve.postinst line 19.
dpkg: error processing package pve-kernel-5.4.128-1-pve (--configure):
 installed pve-kernel-5.4.128-1-pve package post-installation script subprocess returned error exit status 2
dpkg: dependency problems prevent configuration of proxmox-ve:
 proxmox-ve depends on pve-kernel-5.4; however:
  Package pve-kernel-5.4 is not configured yet.

dpkg: error processing package proxmox-ve (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 pve-kernel-5.4.174-2-pve
 pve-kernel-5.4
 pve-kernel-5.4.128-1-pve
 proxmox-ve
E: Sub-process /usr/bin/dpkg returned an error code (1)
 
For anyone looking at this thread, trying to resolve the same problem, I provide a summary below:

The issue not being able to update the kernel is related to having installed kdump-tools while using ZFS as the filesystem.

Check this link describing the bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=934736

The solution is to
either
remove kdump-tools with the command: apt remove kdump-tools
or
comment line 52 using command: nano /etc/kernel/postinst.d/kdump-tools
# sed -e 's/MODULES=.*/MODULES=dep/' /etc/initramfs-tools/initramfs.conf > "$initramfsdir/initramfs.conf" || true
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom