[SOLVED] PVE 8.4 zu 9.1.6 - Upgrade schlägt fehl

974002376

Member
Jun 16, 2023
6
0
6
Hallo zusammen,
ich habe zu Hause meinen Proxmox VE Host von Version 8.4.1 auf 9.1 aktualisiert.
Als Grundlage für das Upgrade habe ich den offiziellen Artikel verwendet: Upgrade von 8 auf 9.

Mein Setup:
  • Proxmox VE auf einer M.2 NVMe installiert
  • Kein ZFS, kein Software-RAID

Die Voraussetzungen für das Upgrade waren erfüllt und der Check hat keine Fehler angezeigt. Warnungen bezüglich der Volumes habe ich übersprungen, da diese zum Zeitpunkt des Upgrades nicht verbunden waren.
Bash:
pve8to9 --full
= CHECKING VERSION INFORMATION FOR PVE PACKAGES =

Checking for package updates..
PASS: all packages up-to-date

Checking proxmox-ve package version..
PASS: already upgraded to Proxmox VE 9

Checking running kernel version..
WARN: unexpected running and installed kernel '6.8.12-19-pve'.

= CHECKING CLUSTER HEALTH/SETTINGS =

SKIP: standalone node.

= CHECKING HYPER-CONVERGED CEPH STATUS =

SKIP: no hyper-converged ceph setup detected!

= CHECKING CONFIGURED STORAGES =

z-pbs01-usb-hdd: error fetching datastores - 500 Can't connect to pbs01.x.y.de:8007 (No route to host)
z-pbs01-usb-hdd1: error fetching datastores - 500 Can't connect to pbs01.x.y.de:8007 (No route to host)
SKIP: storage 'local' disabled.
SKIP: storage 'local-lvm' disabled.
PASS: storage 'lvm-local-ssd-sdb' enabled and active.
PASS: storage 'lvm-local-ssd-sdc' enabled and active.
PASS: storage 'z-local-backup' enabled and active.
WARN: storage 'z-pbs01-usb-hdd' enabled but not active!
WARN: storage 'z-pbs01-usb-hdd1' enabled but not active!
INFO: Checking storage content type configuration..
PASS: no storage content problems found
PASS: no storage reuses a directory for multiple content types.
z-pbs01-usb-hdd1: error fetching datastores - 500 Can't connect to pbs01.x.y.de:8007 (No route to host)
z-pbs01-usb-hdd: error fetching datastores - 500 Can't connect to pbs01.x.y.de:8007 (No route to host)
INFO: Check for usage of native GlusterFS storage plugin...
PASS: No GlusterFS storage found.
INFO: Checking whether all external RBD storages have the 'keyring' option configured
SKIP: No RBD storage configured.

= VIRTUAL GUEST CHECKS =

SKIP: Skipping check for running guests - already upgraded.
INFO: Checking if LXCFS is running with FUSE3 library, if already upgraded..
PASS: systems seems to be upgraded and LXCFS is running with FUSE 3 library
INFO: Checking for VirtIO devices that would change their MTU...
PASS: All guest config descriptions fit in the new limit of 8 KiB
INFO: Checking container configs for deprecated lxc.cgroup entries
PASS: No legacy 'lxc.cgroup' keys found.
INFO: Checking VM configurations for outdated machine versions
PASS: All VM machine versions are recent enough

= MISCELLANEOUS CHECKS =

INFO: Checking common daemon services..
PASS: systemd unit 'pveproxy.service' is in state 'active'
PASS: systemd unit 'pvedaemon.service' is in state 'active'
PASS: systemd unit 'pvescheduler.service' is in state 'active'
PASS: systemd unit 'pvestatd.service' is in state 'active'
INFO: Checking for supported & active NTP service..
PASS: Detected active time synchronisation unit 'chrony.service'
NOTICE: Less than 10 GB free space on root file system.
INFO: Checking if the local node's hostname 'pve01' is resolvable..
INFO: Checking if resolved IP is configured on local node..
PASS: Resolved node IP '192.168.198.2' configured and active on single interface.
INFO: Check node certificate's RSA key size
PASS: Certificate 'pve-root-ca.pem' passed Debian Busters (and newer) security level for TLS connections (4096 >= 2048 RSA)
PASS: Certificate 'pve-ssl.pem' passed Debian Busters (and newer) security level for TLS connections (2048 >= 2048 RSA)
PASS: Certificate 'pveproxy-ssl.pem' passed Debian Busters (and newer) security level for TLS connections (4096 >= 2048 RSA)
INFO: Checking backup retention settings..
PASS: no backup retention problems found.
INFO: checking CIFS credential location..
PASS: no CIFS credentials at outdated location found.
INFO: Checking permission system changes..
INFO: Checking custom role IDs
PASS: no custom roles defined
INFO: Checking node and guest description/note length..
PASS: All node config descriptions fit in the new limit of 64 KiB
INFO: Checking if the suite for the Debian security repository is correct..
PASS: found no suite mismatch
INFO: Checking for existence of NVIDIA vGPU Manager..
PASS: No NVIDIA vGPU Service found.
INFO: Checking bootloader configuration...
PASS: bootloader packages installed correctly
INFO: Check for dkms modules...
SKIP: could not get dkms status
INFO: Check for legacy 'filter' or 'group' sections in /etc/pve/notifications.cfg...
INFO: Check for legacy 'notification-policy' or 'notification-target' options in /etc/pve/jobs.cfg...
PASS: No legacy 'notification-policy' or 'notification-target' options found!
z-pbs01-usb-hdd: error fetching datastores - 500 Can't connect to pbs01.x.y.de:8007 (No route to host)
z-pbs01-usb-hdd1: error fetching datastores - 500 Can't connect to pbs01.x.y.de:8007 (No route to host)
INFO: Check for LVM autoactivation settings on LVM and LVM-thin storages...
SKIP: storage 'local-lvm' (lvmthin) is disabled or inactive
NOTICE: storage 'lvm-local-ssd-sdb' has guest volumes with autoactivation enabled
NOTICE: storage 'lvm-local-ssd-sdc' has guest volumes with autoactivation enabled
NOTICE: Starting with PVE 9, autoactivation will be disabled for new LVM/LVM-thin guest volumes. This system has some volumes that still have autoactivation enabled. All volumes with autoac                                                                                                                                                                                                tivations reside on local storage, where this normally does not cause any issues.
You can run the following command to disable autoactivation for existing LVM/LVM-thin guest volumes:

        /usr/share/pve-manager/migrations/pve-lvm-disable-autoactivation

INFO: Checking lvm config for thin_check_options...
PASS: Check for correct thin_check_options passed
INFO: Check post RRD metrics data format update situation...
INFO: Found '106' RRD files using the old format.
 These are now only used to display historic data. You can delete them if you do not need that.

INFO: Checking for IPAM DB files that have not yet been migrated.
PASS: No legacy IPAM DB found.
PASS: No legacy MAC DB found.
INFO: Checking if the legacy sysctl file '/etc/sysctl.conf' needs to be migrated to new '/etc/sysctl.d/' path.
PASS: Legacy file '/etc/sysctl.conf' is not present.
INFO: Checking if matching CPU microcode package is installed.
PASS: Found matching CPU microcode package 'amd64-microcode' installed.
SKIP: No containers on node detected.

= SUMMARY =

TOTAL:    50
PASSED:   34
SKIPPED:  9
WARNINGS: 3
FAILURES: 0

ATTENTION: Please check the output for detailed information!

Beim Ausführen des Befehls apt dist-upgrade kam es zur folgenden Fehlermeldung:
Bash:
dpkg: error processing package mdadm (--configure):
 installed mdadm package post-installation script subprocess returned error exit status 20
Processing triggers for initramfs-tools (0.148.3) ...
update-initramfs: Generating /boot/initrd.img-6.17.13-1-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
No /etc/kernel/proxmox-boot-uuids found, skipping ESP sync.
Errors were encountered while processing:
 mdadm
Error: Sub-process /usr/bin/dpkg returned an error code (1)

Ein erster Reperaturversuch hat leider keinen Erfolg gebracht:
Bash:
dpkg --configure -a
Setting up mdadm (4.4-11) ...
dpkg: error processing package mdadm (--configure):
 installed mdadm package post-installation script subprocess returned error exit status 20
Processing triggers for initramfs-tools (0.148.3) ...
update-initramfs: Generating /boot/initrd.img-6.17.13-1-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
No /etc/kernel/proxmox-boot-uuids found, skipping ESP sync.
Errors were encountered while processing:
 mdadm
Der Befehl cat /proc/mdstat hat auch keine Einträge ausgegeben.

Daher habe ich mich dazu entschieden, dass Paket zu entfernen:
Bash:
dpkg --remove --force-remove-reinstreq mdadm
(Reading database ... 144906 files and directories currently installed.)
Removing mdadm (4.4-11) ...
update-initramfs: deferring update (trigger activated)
Processing triggers for initramfs-tools (0.148.3) ...
update-initramfs: Generating /boot/initrd.img-6.17.13-1-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
No /etc/kernel/proxmox-boot-uuids found, skipping ESP sync.
Processing triggers for man-db (2.13.1-1) ...

Danach habe ich nochmals apt dist-upgrade ausgeführt.
Bash:
 apt dist-upgrade
The following packages were automatically installed and are no longer required:
  libabsl20220623                 libcbor0.8              liblbfgsb0               libpaper1              libsgutils2-1.46-2  perl-modules-5.36                    python3-pythran
  libavif15                       libdav1d6               libldap-2.5-0            libperl5.36            libsubid4           proxmox-kernel-6.5                   python3-pytz
  libboost-context1.74.0          libdrm-nouveau2         libllvm15                libpmemblk1            libsvtav1enc1       proxmox-kernel-6.5.13-6-pve-signed   python3-tz
  libboost-dev                    libdrm-radeon1          libnl-genl-3-200         libpython3-all-dev     libtext-glob-perl   proxmox-kernel-6.8.12-18-pve-signed  python3.11
  libboost-filesystem1.74.0       libfile-find-rule-perl  libnsl-dev               libpython3.11          libtirpc-dev        python3-all-dev                      python3.11-dev
  libboost-iostreams1.74.0        libflac12               libnumber-compare-perl   libpython3.11-dev      libutempter0        python3-appdirs                      python3.11-minimal
  libboost-program-options1.74.0  libfmt9                 libopenblas-dev          libpython3.11-minimal  libx265-199         python3-beniget                      usrmerge
  libboost-thread1.74.0           libicu72                libopenblas-pthread-dev  libpython3.11-stdlib   libxcb-dri2-0       python3-gast                         xtl-dev
  libboost1.83-dev                libjs-sencha-touch      libopenblas0             librav1e0              libxsimd-dev        python3-ply
Use 'apt autoremove' to remove them.

Summary:
  Upgrading: 0, Installing: 0, Removing: 0, Not Upgrading: 0
root@pve01:/etc/apt/sources.list.d#
root@pve01:/etc/apt/sources.list.d#
root@pve01:/etc/apt/sources.list.d# apt autoremove
REMOVING:
  libabsl20220623                 libcbor0.8              liblbfgsb0               libpaper1              libsgutils2-1.46-2  perl-modules-5.36                    python3-pythran
  libavif15                       libdav1d6               libldap-2.5-0            libperl5.36            libsubid4           proxmox-kernel-6.5                   python3-pytz
  libboost-context1.74.0          libdrm-nouveau2         libllvm15                libpmemblk1            libsvtav1enc1       proxmox-kernel-6.5.13-6-pve-signed   python3-tz
  libboost-dev                    libdrm-radeon1          libnl-genl-3-200         libpython3-all-dev     libtext-glob-perl   proxmox-kernel-6.8.12-18-pve-signed  python3.11
  libboost-filesystem1.74.0       libfile-find-rule-perl  libnsl-dev               libpython3.11          libtirpc-dev        python3-all-dev                      python3.11-dev
  libboost-iostreams1.74.0        libflac12               libnumber-compare-perl   libpython3.11-dev      libutempter0        python3-appdirs                      python3.11-minimal
  libboost-program-options1.74.0  libfmt9                 libopenblas-dev          libpython3.11-minimal  libx265-199         python3-beniget                      usrmerge
  libboost-thread1.74.0           libicu72                libopenblas-pthread-dev  libpython3.11-stdlib   libxcb-dri2-0       python3-gast                         xtl-dev
  libboost1.83-dev                libjs-sencha-touch      libopenblas0             librav1e0              libxsimd-dev        python3-ply

Summary:
  Upgrading: 0, Installing: 0, Removing: 62, Not Upgrading: 0
  Freed space: 1,681 MB

Continue? [Y/n] y

Nach dem Upgrade habe ich das System neu gestartet. Leider lässt sich Proxmox VE seitdem nicht mehr starten. Ich sehe während des Startvorgangs keinen GRUB-Bootloader mehr und lande nach drei Startversuchen direkt im BIOS/UEFI.

Daher habe ich mir die ISO von Proxmox VE 9.1.6 heruntergeladen, auf einen USB-Stick kopiert und diesen am Host angeschlossen. Anschließend habe ich vom USB-Stick gebootet und im Boot-Manager den Eintrag "Advanced Options -> Rescue Boot" ausgewählt. Nach 1-2 Minuten erscheint eine Console. Sowohl direkt über die Console als auch per SSH kann ich mich mit meinen Zugangsdaten anmelden. Auch der Zugriff auf die Web-UI über den FQDN funktioniert problemlos.

Ausgabe von lsblk -o NAME,SIZE,TYPE,FSTYPE:
Bash:
NAME                                                MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS             FSTYPE
sdd                                                   8:48   0 111.8G  0 disk
├─sdd1                                                8:49   0  1007K  0 part
├─sdd2                                                8:50   0   512M  0 part                         vfat
└─sdd3                                                8:51   0 111.3G  0 part                         LVM2_member
  ├─pve-swap                                        252:4    0     8G  0 lvm  [SWAP]                  swap
  ├─pve-root                                        252:5    0  37.8G  0 lvm  /                       ext4
  ├─pve-data_tmeta                                  252:6    0     1G  0 lvm
  │ └─pve-data-tpool                                252:8    0  49.6G  0 lvm
  │   └─pve-data                                    252:9    0  49.6G  1 lvm
  └─pve-data_tdata                                  252:7    0  49.6G  0 lvm
    └─pve-data-tpool                                252:8    0  49.6G  0 lvm
      └─pve-data                                    252:9    0  49.6G  1 lvm

Hat jemand von euch eine Idee, wie ich das Problem beheben kann, sodass der Bootloader GRUB wieder gefunden wird und Proxmox VE normal startet?

Grüße
974002376
 
Last edited:
  • Like
Reactions: Bu66as
Hallo, das Problem liegt an der Meldung in deinem Log:
Code:
No /etc/kernel/proxmox-boot-uuids found, skipping ESP sync.
Die EFI System Partition (sdd2) ist nicht beim proxmox-boot-tool registriert. Deshalb wird der Bootloader beim Upgrade nicht aktualisiert und GRUB startet nach dem Reboot nicht mehr.
Da du bereits per Rescue Boot Zugriff hast, kannst du das so reparieren:
1. ESP initialisieren:
Code:
proxmox-boot-tool init /dev/sdd2
2. Bootloader aktualisieren:
Code:
proxmox-boot-tool refresh
3. Prüfen ob es geklappt hat:
Code:
proxmox-boot-tool status
Danach sollte ein normaler Reboot (ohne USB-Stick) wieder funktionieren.
Falls proxmox-boot-tool init einen Fehler wirft, vorher die ESP formatieren:
Code:
proxmox-boot-tool format /dev/sdd2
proxmox-boot-tool init /dev/sdd2
Siehe auch: https://pve.proxmox.com/wiki/Host_Bootloader#sysboot_proxmox_boot_tool
 
  • Like
Reactions: ThoSo and 974002376
Hallo @Bu66as,
Danke für die Kurzanleitung.

Nachstehend die von mir ausgeführten Befehle inkl. deren Ausgabe:
Bash:
root@pve01:~# proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
E: /etc/kernel/proxmox-boot-uuids does not exist.
root@pve01:~# proxmox-boot-tool init /dev/sdd2
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
UUID="2696-0652" SIZE="536870912" FSTYPE="vfat" PARTTYPE="c12a7328-f81f-11d2-ba4b-00a0c93ec93b" PKNAME="sdd" MOUNTPOINT=""
Mounting '/dev/sdd2' on '/var/tmp/espmounts/2696-0652'.
Installing grub x86_64 target..
Installing for x86_64-efi platform.
Installation finished. No error reported.
Installing grub x86_64 target (removable)..
Installing for x86_64-efi platform.
Installation finished. No error reported.
Unmounting '/dev/sdd2'.
Adding '/dev/sdd2' to list of synced ESPs..
Refreshing kernels and initrds..
Running hook script 'proxmox-auto-removal'..
Running hook script 'zz-proxmox-boot'..
No /etc/kernel/cmdline found - falling back to /proc/cmdline
Copying and configuring kernels on /dev/disk/by-uuid/2696-0652
        Copying kernel 6.17.13-1-pve
        Copying kernel 6.8.12-19-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.17.13-1-pve
Found initrd image: /boot/initrd.img-6.17.13-1-pve
Found linux image: /boot/vmlinuz-6.8.12-19-pve
Found initrd image: /boot/initrd.img-6.8.12-19-pve
Adding boot menu entry for UEFI Firmware Settings ...
done
root@pve01:~# proxmox-boot-tool refresh
Running hook script 'proxmox-auto-removal'..
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
No /etc/kernel/cmdline found - falling back to /proc/cmdline
Copying and configuring kernels on /dev/disk/by-uuid/2696-0652
        Copying kernel 6.17.13-1-pve
        Copying kernel 6.8.12-19-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.17.13-1-pve
Found initrd image: /boot/initrd.img-6.17.13-1-pve
Found linux image: /boot/vmlinuz-6.8.12-19-pve
Found initrd image: /boot/initrd.img-6.8.12-19-pve
Adding boot menu entry for UEFI Firmware Settings ...
done
root@pve01:~# proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with uefi
2696-0652 is configured with: grub (versions: 6.17.13-1-pve, 6.8.12-19-pve)
Danach Server neu gestartet und PVE fährt wieder wie gewohnt wieder hoch. Vielen lieben Dank!

Mir ist aktuell noch nicht klar, wie der Fehler entstanden ist. Eventuell habe ich bei einem Upgrade oder Update von PVE etwas übersehen?!