Upgrade from 4.x to 5.0 not booting

[Idar Lund]

I've followed the guide here; https://pve.proxmox.com/wiki/Upgrade_from_4.x_to_5.0
After the upgrade, the servers wont boot with the newest kernel (4.10.17-1-pve), however it will boot into 5.0-23 with the old kernel (4.4.67-1-pve).

The loading screens tells me;

Loading Linux 4.10.17-1-pve ...
Loading initial ramdisk ...

And then it freezes.

I've tried adding "verbose" and "debug" to the boot line, but still the same result as above.

Any suggestions on how I can troubleshoot this?

 

[pabernethy]

That sounds like the dist-upgrade didn't complete successfully. Are you certain it did? What files do you have in /boot? You could just boot the old kernel and re-run the upgrade process.

 

[Idar Lund]

Yes, everything is updated;

root@server2:~# apt-get update && apt-get dist-upgrade
Get:1 hxxp://download.proxmox.com/debian stretch InRelease [3,058 B]
Get:2 hxxp://download.proxmox.com/debian/ceph-luminous stretch InRelease [2,914 B]
Ign:3 hxxp://ftp.no.debian.org/debian stretch InRelease
Hit:4 hxxp://ftp.no.debian.org/debian stretch Release
Get:5 hxxp://security.debian.org stretch/updates InRelease [62.9 kB]
Get:7 hxxp://security.debian.org stretch/updates/main amd64 Packages [127 kB]
Get:8 hxxp://security.debian.org stretch/updates/main Translation-en [51.9 kB]
Fetched 247 kB in 1s (209 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

(I had to change http to hxxp because the forum didn't accept links)

And the /boot directory does have the necessary files;

root@server2:~# ls -laFh /boot/*4.4.67*
-rw-r--r-- 1 root root 186K Jun 23 09:16 /boot/config-4.4.67-1-pve
-rw-r--r-- 1 root root  27M Jul 30 18:57 /boot/initrd.img-4.4.67-1-pve
-rw-r--r-- 1 root root 3.8M Jun 23 09:16 /boot/System.map-4.4.67-1-pve
-rw-r--r-- 1 root root 6.9M Jun 23 09:16 /boot/vmlinuz-4.4.67-1-pve
root@server2:~# ls -laFh /boot/*4.10.17*
-rw-r--r-- 1 root root 200K Jul 11 09:55 /boot/config-4.10.17-1-pve
-rw-r--r-- 1 root root  29M Jul 30 19:31 /boot/initrd.img-4.10.17-1-pve
-rw-r--r-- 1 root root 3.6M Jul 11 09:55 /boot/System.map-4.10.17-1-pve
-rw-r--r-- 1 root root 7.4M Jul 11 09:55 /boot/vmlinuz-4.10.17-1-pve

I believe there's a problem with the 4.10 kernel/modules with my specific hardware. How can I troubleshoot this when debug and verbose does not work?

 

[pabernethy]

They should work. Be aware that there should only be one parameter regarding logging on the kernel line. Otherwise the last parameter takes precedence. So if the kernel line in grub is
linux /boot/vmlinuz-4.10.17-1-pve root=/dev/mapper/pve-root ro debug quiet
the resulting output will be quiet.

 

[Idar Lund]

The line I have used, which should give the most output is;

linux   /boot/vmlinuz-4.10.17-1-pve root=/dev/mapper/pve-root ro  verbose

However, the system still just freezes at;

Loading Linux 4.10.17-1-pve ...
Loading initial ramdisk ...

 

[pabernethy]

And I assume the initial ramdisk is given by

initrd  /boot/initrd.img-4.10.17-1-pve

 

[Idar Lund]

Yes. Had to change the filename to txt, since cfg was not allowed on the forum.

 

[pabernethy]

Please try reinstalling the kernel and regenerating the initial RamFS with

# apt-get install --reinstall pve-kernel-4.10.17-1-pve

and post the output.

 

[Idar Lund]

Did the reinstall, but the server(s) will still not boot.
Here's the reinstall output;

root@server2:~# apt-get --reinstall install pve-kernel-4.10.17-1-pve
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded.
Need to get 0 B/48.0 MB of archives.
After this operation, 0 B of additional disk space will be used.
(Reading database ... 111354 files and directories currently installed.)
Preparing to unpack .../pve-kernel-4.10.17-1-pve_4.10.17-16_amd64.deb ...
Unpacking pve-kernel-4.10.17-1-pve (4.10.17-16) over (4.10.17-16) ...
Setting up pve-kernel-4.10.17-1-pve (4.10.17-16) ...
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.10.17-1-pve /boot/vmlinuz-4.10.17-1-pve
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.10.17-1-pve /boot/vmlinuz-4.10.17-1-pve
update-initramfs: Generating /boot/initrd.img-4.10.17-1-pve
run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 4.10.17-1-pve /boot/vmlinuz-4.10.17-1-pve
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.10.17-1-pve /boot/vmlinuz-4.10.17-1-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.10.17-1-pve
Found initrd image: /boot/initrd.img-4.10.17-1-pve
Found linux image: /boot/vmlinuz-4.4.67-1-pve
Found initrd image: /boot/initrd.img-4.4.67-1-pve
Found linux image: /boot/vmlinuz-4.4.59-1-pve
Found initrd image: /boot/initrd.img-4.4.59-1-pve
Found linux image: /boot/vmlinuz-4.4.49-1-pve
Found initrd image: /boot/initrd.img-4.4.49-1-pve
Found linux image: /boot/vmlinuz-4.4.40-1-pve
Found initrd image: /boot/initrd.img-4.4.40-1-pve
Found linux image: /boot/vmlinuz-4.4.35-1-pve
Found initrd image: /boot/initrd.img-4.4.35-1-pve
Found linux image: /boot/vmlinuz-4.4.24-1-pve
Found initrd image: /boot/initrd.img-4.4.24-1-pve
Found linux image: /boot/vmlinuz-4.4.21-1-pve
Found initrd image: /boot/initrd.img-4.4.21-1-pve
Found linux image: /boot/vmlinuz-4.4.19-1-pve
Found initrd image: /boot/initrd.img-4.4.19-1-pve
Found linux image: /boot/vmlinuz-4.4.16-1-pve
Found initrd image: /boot/initrd.img-4.4.16-1-pve
Found linux image: /boot/vmlinuz-4.4.6-1-pve
Found initrd image: /boot/initrd.img-4.4.6-1-pve
Found memtest86+ image: /boot/memtest86+.bin
Found memtest86+ multiboot image: /boot/memtest86+_multiboot.bin
done

Instead of trying stuff like this. How can I troubleshoot the boot sequence? It's obvious that there's a problem with my kernel/initrd and my hardware. How can I get some debug information here so that I'm able to figure out what's not working?

 

[pabernethy]

You can try booting from the 5.0 ISO. If it boots it shouldn't be a general hardware incompatibility. In that case the course of action would be backup and clean reinstall.

 

[Idar Lund]

After the release of pve51, i upgraded and got new kernel (4.13). The servers now boot up normally with 4.13.4-26. I don't know what was wrong in the 4.10 kernel, but fortunately the new 4.13 is now working!