Migrated Ubuntu guests take a long time to start up

[budy]

I have migrated some guests from my OracleVM (Xen) to Proxmox. Amongst them are two Ubuntu 18.0.4 guests, which started up under Xen in, well… usual speeds. However, starting these guests under KVM/Proxmox causes them to take a long time (up to 150s) to begin the actual boot process, such as that something is written to the console window.

Before that, these guest show a high cpu load before then dropping into actually booting the system. Is there an option to debug that? I'd really like to know, what's going one with those guests and get the regular startup speed restored.

 

[Stoiko Ivanov]

this depends on where exactly the boot hangs - does it take 150s before the boot-loader shows up? - then check the logs of the start task and the journal on the host
does it take this long after the guests boot-loader shows up - then try editing the kernel command line and remove the quiet flag
and check the journal on the guest for where the time is spent

I hope this helps!

 

[budy]

Thanks for the suggestions. I checked the boot arguments, but the quiet flag isn't even present. However, when I managed to get to the screen, where I can choose which kernel to load the boot process hangs at "Loading initial ramdisk". This is what seems to take a long time. Afterwards the system boots up just as fast as I remember it to be.

I also looked at the journal but that only seems to start where the kernel gets loaded so the lapsed time before seems to be unaccounted for.

P.S. Phew… I just had to rollback the guest's vdisk, due to an issue when upgrading Ubuntu and I was astonished, that the next reboot took only seconds… I immediately performed another restart from within the guest and monitored my Ceph cluster and it looks as if the initrd gets loaded with rather kb/s - this is what take so long. After the kernel has started the transfers seem to be just fine.

 

[Stoiko Ivanov]

"Loading initial ramdisk".

hmm - this is where my system 'hangs' because the kernel is initializing - can you post the content of '/proc/cmdline'?

 

[budy]

From which host? The guest (after ithas started) or the Proxmox-Server?

The guest's output goes like this:

root@privacyidea:~# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-4.15.0-88-generic root=/dev/mapper/privacyidea--vg-root ro

Cheers,
budy

 

[Stoiko Ivanov]

you can maybe find the missing time - from the journal:
`journalctl -b` gives you the log since booting - look for large gaps in the timestamps

I hope this helps!

 

[budy]

No, unfortuanetly, this doesn't shed any light on whats going on. I have noted the time when the guest had startet - 08.57:02 and when the Konsole showed the kernel booting - 08:58:02. When the guest became ready I logged in and issued a journalctl -b and that went like this:

-- Logs begin at Sun 2020-01-26 15:27:11 CET, end at Tue 2020-03-03 09:00:04 CET. --
Mär 03 08:58:06 privacyidea kernel: Linux version 4.15.0-88-generic (buildd@lgw01-amd64-036) (gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)) #88-Ubuntu SMP Tue Feb 11 20:11:34 UTC 2020 (Ubuntu 4.15.0-88.88-generic 4.15.18)
Mär 03 08:58:06 privacyidea kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-88-generic root=/dev/mapper/privacyidea--vg-root ro
Mär 03 08:58:06 privacyidea kernel: KERNEL supported cpus:

So there's something going on even before the kernel starts.