[SOLVED] Pve-kernel-helper won't install

jaxom_99 · May 28, 2021

Hello fellow PVE users o/

I'm running PVE on a single node, and I had some update troubles (probably due to misusing "apt upgrade" command at some point). I am in the following situation : proxmox is not correctly installed, because pve-kernel-helper is missing. However, I can't install or fix it, because my system refuses to divert the /usr/sbin/grub-install file, that also belongs to "grub2-common".
See logs below.

Can anybody help me get back on my foots ?

I'm trying to avoid a clean install, since I have some production services on this server.
Cheers,
-- Jaxom

#pveversion -v

Code:

proxmox-ve: not correctly installed (running kernel: 5.4.78-2-pve)
pve-manager: 6.4-6 (running version: 6.4-6/be2fa32c)
pve-kernel-5.4.78-2-pve: 5.4.78-2
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.2-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve3
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.0.3-1
libpve-access-control: 6.4-1
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-3
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-2
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.1.6-2
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.5-5
pve-cluster: 6.4-1
pve-container: 3.3-5
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.2-3
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-6
pve-xtermjs: 4.7.0-3
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.4-pve1

# pveupgrade

Code:

Starting system upgrade: apt-get dist-upgrade
Lecture des listes de paquets... Fait
Construction de l'arbre des dépendances       
Lecture des informations d'état... Fait
Vous pouvez lancer « apt --fix-broken install » pour corriger ces problèmes.
Les paquets suivants contiennent des dépendances non satisfaites :
 proxmox-ve : Dépend: pve-kernel-helper mais il n'est pas installé
E: Dépendances non satisfaites. Essayez « apt --fix-broken install » sans paquet
   (ou indiquez une solution).

Your System is up-to-date

#apt --fix-broken install

Code:

Lecture des listes de paquets... Fait
Construction de l'arbre des dépendances       
Lecture des informations d'état... Fait
Correction des dépendances... Fait
Les paquets suivants ont été installés automatiquement et ne sont plus nécessaires :
  efibootmgr mokutil shim-helpers-amd64-signed shim-signed-common shim-unsigned
Veuillez utiliser « sudo apt autoremove » pour les supprimer.
Les paquets supplémentaires suivants seront installés :
  pve-kernel-helper
Les NOUVEAUX paquets suivants seront installés :
  pve-kernel-helper
0 mis à jour, 1 nouvellement installés, 0 à enlever et 0 non mis à jour.
5 partiellement installés ou enlevés.
Il est nécessaire de prendre 0 o/11,3 ko dans les archives.
Après cette opération, 60,4 ko d'espace disque supplémentaires seront utilisés.
Souhaitez-vous continuer ? [O/n] o
(Lecture de la base de données... 65792 fichiers et répertoires déjà installés.)
Préparation du dépaquetage de .../pve-kernel-helper_6.4-2_all.deb ...
Dépaquetage de pve-kernel-helper (6.4-2) ...
dpkg: erreur de traitement de l'archive /var/cache/apt/archives/pve-kernel-helper_6.4-2_all.deb (--unpack) :
 tentative de remplacement de « /usr/sbin/grub-install », qui appartient aussi au paquet grub2-common 2.02+dfsg1-18-pve1
Pas de détournement « détournement de /usr/sbin/grub-install en /usr/sbin/grub-install.real par pve-kernel-helper », aucun supprimé.
Des erreurs ont été rencontrées pendant l'exécution :
 /var/cache/apt/archives/pve-kernel-helper_6.4-2_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

Moayad · May 28, 2021

Hi,

Please send the full output of the apt update command also please try the dpkg --configure -a

Is the /etc/hosts include the 127.0.0.1 ?

Stoiko Ivanov · May 28, 2021

Hm - we added the diversion quite recently - maybe this might be something we could improve ...

In addition to what @Moayad suggested - could you please also post:
* `dpkg -l |grep -Ei 'kernel|grub'`
* `ls -la /usr/sbin/grub*`

also any hint to when and how you ran `apt upgrade` would be helpful - maybe we can then reproduce the situation here

jaxom_99 · May 28, 2021

Thanks for the quick feedback. Unfortunately I did not log precisely the history of commands that led me in the situation :/ Let's try and remember :

1° I made a standard install over Debian buster last year, ran some upgrades and reboots irregularly (and surely made mistakes).
2° The first encountered error was a lack of web interface, and /etc/pve was empty. So pveversion returned "command not found" (but my VMs were still running), so I called "apt update" and "apt upgrade", that ouput some errors
3° Following this thread, I removed and restored the grub-common and grub-pc packages, and had to reset some GRUB settings on my drives, but the "apt upgrade" still mentionned errors
4° pveversion output was "not correctly installed" for all components, so I ran dpkg --configure -a which mentionned also a few errors.
5° with fear for my GRUB (I cross fingers not to reboot yet), I had to temporarily remove /etc/grub.d/000_proxmox_boot_header to run update-grub.
6° Then I tried to isolate the error, and traced it to the situation in my first post.

Here are the outputs asked, thanks again for your help whatsoever !

* apt update

Code:

Réception de :1 http://security.debian.org/debian-security buster/updates InRelease [65,4 kB]                 
Atteint :2 http://mirrors.ircam.fr/pub/debian buster InRelease                                               
Réception de :3 http://mirrors.ircam.fr/pub/debian buster-updates InRelease [51,9 kB]                     
Réception de :4 http://security.debian.org/debian-security buster/updates/main Sources [186 kB]
Réception de :5 http://security.debian.org/debian-security buster/updates/main amd64 Packages [290 kB]
Atteint :6 http://download.proxmox.com/debian/pve buster InRelease                         
593 ko réceptionnés en 11s (55,6 ko/s)                             
Lecture des listes de paquets... Fait
Construction de l'arbre des dépendances       
Lecture des informations d'état... Fait
Tous les paquets sont à jour.

* `dpkg -l |grep -Ei 'kernel|grub'

Code:

ii  dmeventd                             2:1.02.155-pve4                         amd64        Linux Kernel Device Mapper event daemon
ii  dmsetup                              2:1.02.155-pve4                         amd64        Linux Kernel Device Mapper userspace library
ii  grub-common                          2.02+dfsg1-18-pve1                      amd64        GRand Unified Bootloader (common files)
iF  grub-pc                              2.02+dfsg1-18-pve1                      amd64        GRand Unified Bootloader, version 2 (PC/BIOS version)
ii  grub-pc-bin                          2.02+dfsg1-18-pve1                      amd64        GRand Unified Bootloader, version 2 (PC/BIOS modules)
ii  grub2-common                         2.02+dfsg1-18-pve1                      amd64        GRand Unified Bootloader (common files for version 2)
ii  ipset                                6.38-1.2                                amd64        administration tool for kernel IP sets
ii  kmod                                 26-1                                    amd64        tools for managing Linux kernel modules
ii  libaio1:amd64                        0.3.112-3                               amd64        Linux kernel AIO access library - shared library
ii  libdevmapper-event1.02.1:amd64       2:1.02.155-pve4                         amd64        Linux Kernel Device Mapper event support library
ii  libdevmapper1.02.1:amd64             2:1.02.155-pve4                         amd64        Linux Kernel Device Mapper userspace library
ii  libdrm-amdgpu1:amd64                 2.4.97-1                                amd64        Userspace interface to amdgpu-specific kernel DRM services -- runtime
ii  libdrm-common                        2.4.97-1                                all          Userspace interface to kernel DRM services -- common files
ii  libdrm-intel1:amd64                  2.4.97-1                                amd64        Userspace interface to intel-specific kernel DRM services -- runtime
ii  libdrm-nouveau2:amd64                2.4.97-1                                amd64        Userspace interface to nouveau-specific kernel DRM services -- runtime
ii  libdrm-radeon1:amd64                 2.4.97-1                                amd64        Userspace interface to radeon-specific kernel DRM services -- runtime
ii  libdrm2:amd64                        2.4.97-1                                amd64        Userspace interface to kernel DRM services -- runtime
ii  libzmq5:amd64                        4.3.1-4+deb10u2                         amd64        lightweight messaging kernel (shared library)
ii  proxmox-backup-restore-image         0.2.1                                   amd64        Kernel/initramfs images for Proxmox Backup single-file restore.
ii  pve-firmware                         3.2-3                                   all          Binary firmware code for the pve-kernel
iU  pve-kernel-5.4                       6.4-2                                   all          Latest Proxmox VE Kernel Image
iF  pve-kernel-5.4.114-1-pve             5.4.114-1                               amd64        The Proxmox PVE Kernel Image
ii  pve-kernel-5.4.78-2-pve              5.4.78-2                                amd64        The Proxmox PVE Kernel Image
ic  pve-kernel-helper                    6.4-2                                   all          Function for various kernel maintenance tasks.
ii  rsyslog                              8.1901.0-1                              amd64        reliable system and kernel logging daemon

* ls -la /usr/sbin/grub*

Code:

lrwxrwxrwx 1 root root      35 mai   23  2019 /usr/sbin/grub-bios-setup -> ../lib/grub/i386-pc/grub-bios-setup
-rwxr-xr-x 1 root root 1036872 mai   23  2019 /usr/sbin/grub-install
-rwxr-xr-x 1 root root  804664 mai   23  2019 /usr/sbin/grub-macbless
-rwxr-xr-x 1 root root    8020 mai   23  2019 /usr/sbin/grub-mkconfig
-rwxr-xr-x 1 root root  213400 mai   23  2019 /usr/sbin/grub-mkdevicemap
-rwxr-xr-x 1 root root  817240 mai   23  2019 /usr/sbin/grub-probe
-rwxr-xr-x 1 root root    4109 mai   23  2019 /usr/sbin/grub-reboot
-rwxr-xr-x 1 root root    3559 mai   23  2019 /usr/sbin/grub-set-default

* dpkg --configure -a

Code:

Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 5.4.114-1-pve /boot/vmlinuz-5.4.114-1-pve
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 5.4.114-1-pve /boot/vmlinuz-5.4.114-1-pve
update-initramfs: Generating /boot/initrd.img-5.4.114-1-pve
run-parts: executing /etc/kernel/postinst.d/proxmox-auto-removal 5.4.114-1-pve /boot/vmlinuz-5.4.114-1-pve
/etc/kernel/postinst.d/proxmox-auto-removal: 4: .: Can't open /usr/share/pve-kernel-helper/scripts/functions
run-parts: /etc/kernel/postinst.d/proxmox-auto-removal exited with return code 127
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/pve-kernel-5.4.114-1-pve.postinst line 19.
dpkg: erreur de traitement du paquet pve-kernel-5.4.114-1-pve (--configure) :
 installed pve-kernel-5.4.114-1-pve package post-installation script subprocess returned error exit status 2
Installation pour la plate-forme i386-pc.
grub-install : attention : cette étiquette de partition GPT ne contient pas de partition d'amorçage BIOS ; l'embarquage ne sera pas possible.
grub-install : erreur : l'embarquage est impossible, il est pourtant nécessaire pour les installations interdisques.
Installation pour la plate-forme i386-pc.
Installation terminée, sans erreur.
Installation pour la plate-forme i386-pc.
grub-install : attention : Le système de fichiers « ext2 » ne prend pas en charge l'embarquage.
grub-install : attention : L'embarquage est impossible. GRUB ne peut être installé sur cette configuration qu'en utilisant les listes de blocs. Cependant, les listes de blocs ne sont PAS fiables et leur utilisation est déconseillée..
Installation terminée, sans erreur.
Paramétrage de initramfs-tools (0.133+deb10u1) ...
update-initramfs: deferring update (trigger activated)
Paramétrage de pve-kernel-5.4.114-1-pve (5.4.114-1) ...
Paramétrage de grub-pc (2.02+dfsg1-18-pve1) ...
Traitement des actions différées (« triggers ») pour initramfs-tools (0.133+deb10u1) ...
update-initramfs: Generating /boot/initrd.img-5.4.114-1-pve

# Splashscreen for Grub errors on ext2 drives

Création du fichier de configuration GRUB…                                                                                                           
/etc/grub.d/000_proxmox_boot_header: 4: .: Can't open /usr/share/pve-kernel-helper/scripts/functions
dpkg: erreur de traitement du paquet grub-pc (--configure) :
 installed grub-pc package post-installation script subprocess returned error exit status 127
dpkg: des problèmes de dépendances empêchent la configuration de proxmox-ve :
 proxmox-ve dépend de pve-kernel-helper ; cependant :
  Le paquet pve-kernel-helper n'est pas installé.

dpkg: erreur de traitement du paquet proxmox-ve (--configure) :
 problèmes de dépendances - laissé non configuré
dpkg: des problèmes de dépendances empêchent la configuration de pve-kernel-5.4 :
 pve-kernel-5.4 dépend de pve-kernel-5.4.114-1-pve ; cependant :
 Le paquet pve-kernel-5.4.114-1-pve n'est pas encore configuré.

dpkg: erreur de traitement du paquet pve-kernel-5.4 (--configure) :
 problèmes de dépendances - laissé non configuré
Traitement des actions différées (« triggers ») pour initramfs-tools (0.133+deb10u1) ...
update-initramfs: Generating /boot/initrd.img-5.4.114-1-pve
/etc/initramfs/post-update.d//proxmox-boot-sync: 10: /etc/initramfs/post-update.d//proxmox-boot-sync: /usr/sbin/proxmox-boot-tool: not found
run-parts: /etc/initramfs/post-update.d//proxmox-boot-sync exited with return code 127
dpkg: erreur de traitement du paquet initramfs-tools (--configure) :
 installed initramfs-tools package post-installation script subprocess returned error exit status 1
Des erreurs ont été rencontrées pendant l'exécution :
 pve-kernel-5.4.114-1-pve
 grub-pc
 proxmox-ve
 pve-kernel-5.4
 initramfs-tools

Stoiko Ivanov · May 28, 2021

On a hunch - try initially purging `pve-kernel-helper` - it's config-files (or rather files in /etc) are still around, and they fails (since it's other files are not there any more):

Code:

apt purge pve-kernel-helper

don't continue if it tries to remove other packages as well

if this fails then try moving the config-files:

Code:

/etc/grub.d/000_proxmox_boot_header
/etc/initramfs/post-update.d/proxmox-boot-sync
/etc/kernel/postinst.d/proxmox-auto-removal
/etc/kernel/postinst.d/zz-proxmox-boot
/etc/kernel/postrm.d/proxmox-auto-removal
/etc/kernel/postrm.d/zz-proxmox-boot

out of the way (by copying them to another directory and removing them from /etc) and then try the reinstalling, configuration steps you did before again.

do you have a `/etc/kernel/proxmox-boot-uuids` or `/etc/kernel/pve-efiboot-uuids` file ?

I hope this helps!

jaxom_99 · May 30, 2021

Thank you Stoiko for your inputs. It seems the apt purge is not effective, all I see is the same error . I then tried to remove the files you mention, and re-run the commands (update, dist-upgrade, --fix-broken install, even purge/reinstall the grub parts), but I'm stuck with the same output : apt refuses to install pve-kernel-heler, with this problem of diverting the file. It seems to have blocked any use of apt at all, as I can't install other ackages either !

It turns out I don't have any of the /etc/kernel files you mention. Should I worry much ?

I stay available if you have some other ideas, or if you want to investigate this further.

jaxom_99 · Jun 7, 2021

New week, same troubles : even when apt finds new updates to apply, it still is stuck by this "unapplicable" diversion of /usr/sbin/grub-install.real by pve-kernel-helper.
@Stoiko Ivanov should I open a ticket somewhere ? Or maybe there is already one ? Thanks again for your help so far.

Stoiko Ivanov · Jun 8, 2021

I think I managed to partially reproduce your issue (running apt remove pve-kernel-helper, and thus having to manually touch /please-remove-proxmox-ve) - but here running:
`apt purge pve-kernel-helper`
`apt install pve-kernel-helper`
fixed the situation

could you maybe post the output (while setting to an english locale (my french is good enough to get the general idea of the error - but in that case the literal error-message would help a lot) of the two commands above?

Stoiko Ivanov · Jun 8, 2021

Also if this still does not help - maybe try manually adding the diversion before installing pve-kernel-helper:
`dpkg-divert --package pve-kernel-helper --add --rename --divert /usr/sbin/grub-install.real /usr/sbin/grub-install`
(again post the output of the command please)

jaxom_99 · Jun 9, 2021

Sure thing, should have switched locales form the start. Here are the error messages in Shakespeare's words :
* apt purge :

Code:

Reading package lists... Done
Building dependency tree      
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 proxmox-ve : Depends: pve-kernel-helper but it is not going to be installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

* apt install

Code:

Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following packages were automatically installed and are no longer required:
  efibootmgr mokutil shim-helpers-amd64-signed shim-signed-common shim-unsigned
Use 'sudo apt autoremove' to remove them.
The following NEW packages will be installed:
  pve-kernel-helper
0 upgraded, 1 newly installed, 0 to remove and 8 not upgraded.
1 not fully installed or removed.
Need to get 0 B/11.3 kB of archives.
After this operation, 60.4 kB of additional disk space will be used.
(Reading database ... 65792 files and directories currently installed.)
Preparing to unpack .../pve-kernel-helper_6.4-2_all.deb ...
Unpacking pve-kernel-helper (6.4-2) ...
dpkg: error processing archive /var/cache/apt/archives/pve-kernel-helper_6.4-2_all.deb (--unpack):
 trying to overwrite '/usr/sbin/grub-install', which is also in package grub2-common 2.02+dfsg1-18-pve1
No diversion 'diversion of /usr/sbin/grub-install to /usr/sbin/grub-install.real by pve-kernel-helper', none removed.
Errors were encountered while processing:
 /var/cache/apt/archives/pve-kernel-helper_6.4-2_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

However, running the diversion manually seemed to work ! It returns : Adding 'diversion of /usr/sbin/grub-install to /usr/sbin/grub-install.real by pve-kernel-helper'.

I then ran apt update and dist-upgrade : all is up to date ! I rebooted with success, and my VMs are running smoothly.

So thank you very much for looking into it !

PS: I don't have the money yet for a proxmox subscription, but I'm looking into it as a way to support your efforts

TiMMaY · Jun 24, 2021

Just FYI - Still an issue

Used the diversion as previously mentioned and all is well

TY!

fabian · Jun 24, 2021

from both of you it would be interesting to get the /var/log/apt/term.log and /var/log/apt/history.log contents (or their rotated variants, if rotation already happened). please include the command before the first time it fails, and the first time it fails (at least). thanks!

TiMMaY · Jun 24, 2021

fabian said:
from both of you it would be interesting to get the /var/log/apt/term.log and /var/log/apt/history.log contents (or their rotated variants, if rotation already happened). please include the command before the first time it fails, and the first time it fails (at least). thanks!

I've been configuring the new server so doubt this will help... but here goes...

fabian · Jun 24, 2021

thanks - but yeah, we'd need those from the system that had the error

TiMMaY · Jun 24, 2021

fabian said:
thanks - but yeah, we'd need those from the system that had the error

it was

TiMMaY · Jun 24, 2021

Not sure what happened first go around but works for me this time lol

Made a script to help automate the install if anyone can use this:

#!/bin/bash
wget -qO - http://download.proxmox.com/debian/proxmox-ve-release-6.x.gpg | apt-key add -
echo "deb http://download.proxmox.com/debian/pve buster pve-no-subscription" | tee /etc/apt/sources.list.d/pve-install-repo.list
echo "deb http://download.proxmox.com/debian/ceph-nautilus buster main" | tee /etc/apt/sources.list.d/ceph.list

apt update && apt dist-upgrade
apt update 

apt install ifupdown2 net-tools vim proxmox-ve postfix open-iscsi -y

sed 's/^/#/' /etc/apt/sources.list.d/pve-enterprise.list
init 6

jaxom_99 · Jun 28, 2021

fabian said:
from both of you it would be interesting to get the /var/log/apt/term.log and /var/log/apt/history.log contents (or their rotated variants, if rotation already happened). please include the command before the first time it fails, and the first time it fails (at least). thanks!

Okay, I think I found them ! Sorry it's in french, "diversion" vould be "détournement".
Might be a "apt upgrade" then a "apt auto-remove" that landed me here...

fabian · Jun 28, 2021

okay, so you somehow managed to accidentally uninstall half of PVE, and something is not handled correctly there and breaks the system. thanks for the log - this should help make the handling more robust in those cases as well (which should never happen for a regular upgrade anyway!).

jaxom_99 · Jun 28, 2021

fabian said:
okay, so you somehow managed to accidentally uninstall half of PVE, and something is not handled correctly there and breaks the system. thanks for the log - this should help make the handling more robust in those cases as well (which should never happen for a regular upgrade anyway!).

>.<" PEBCAK is definitely the hardest bug to smash !
And yet the VMs kept spinning, no trouble noticed. Thanks again for your time and work

[SOLVED] Pve-kernel-helper won't install

New Member

Proxmox Staff Member

Proxmox Staff Member

New Member

Proxmox Staff Member

New Member

New Member

Proxmox Staff Member

Proxmox Staff Member

New Member

New Member

Proxmox Staff Member

New Member

Attachments

Proxmox Staff Member

New Member

New Member

New Member

Attachments

Proxmox Staff Member

New Member