6.4 > 7.0.x stuck in apt dist-upgrade after shell disconnect

[fuzzyduck]

Hello! just registered to ask you for a solution to this problem.

After doing the proper prerequisites to go from 6.4 to 7.0.x i issued the apt dist-upgrade command from the web shell.
After pressing yes to continue I then ended up on a rather long text about kernels, about being able to return to a previous one?
It gave me an indefinite prompt without a timer to continue

After that i made a stupid mistake of disconnecting wifi so now i lost the terminal screen to continue.(yes wifi :/)

Now i got a lock on /var/lib/dpkg/lock-frontend obviously and ps aux | grep -i apt gives me:

root      40431  0.0  0.0  68824 55204 pts/1    S+   Jul10   0:00 apt dist-upgrade

A. is there a way to return to that still running web-shell? i still have it running in the tasks.

or

B. Kill the apt? then what? since im in the beginning of the long upgrade to version 7.0.x. I dont think ANY packages are configured yet, that text was the first popup after pressing yes to continue on the dist-upgrade

i dont want to reboot just yet of kill the apt process, since it might mess up entirely

thank you!

 

[oguz]

hi,

After doing the proper prerequisites to go from 6.4 to 7.0.x i issued the apt dist-upgrade command from the web shell.

you shouldn't have done that as the upgrade guide [0] says:

Perform the actions via console or ssh; preferably via console to avoid interrupted ssh connections. Do not carry out the upgrade when connected via the virtual console offered by the GUI; as this will get interrupted during the upgrade.

A. is there a way to return to that still running web-shell? i still have it running in the tasks.

no, unless you started a tmux/screen session beforehand...

B. Kill the apt? then what? since im in the beginning of the long upgrade to version 7.0.x. I dont think ANY packages are configured yet, that text was the first popup after pressing yes to continue on the dist-upgrade

depending on where it was, it might be fine to kill it. to be sure you can check your /var/log/apt/history.log file to see if any package was upgraded.
if you're sure there wasn't anything going on then you could kill the process and start over (this time with SSH )


[0]: https://pve.proxmox.com/wiki/Upgrade_from_6.x_to_7.0#Actions_step-by-step

 

[fuzzyduck]

thank you for your reply! yes im verrrry aware of my webshell error i made

Can anyone confirm the kernel-text is the very first you see that needs manual acknowledgement when doing the dist-upgrade?

 

[fuzzyduck]

the apt history log from that date and time:

Start-Date: 2021-07-10  20:37:51
Commandline: apt dist-upgrade
Install: pve-kernel-5.4.124-1-pve:amd64 (5.4.124-1, automatic)
Upgrade: proxmox-widget-toolkit:amd64 (2.5-6, 2.6-1), pve-kernel-5.4:amd64 (6.4-2, 6.4-4), libgnutlsxx28:amd64 (3.6.7-4+deb10u6, 3.6.7-4+deb10u7), proxmox-backup-file-restore:amd64 (1.1.8-1, 1.1.10-1), libpve-access-control:amd64 (6.4-1, 6.4-3), linux-libc-dev:amd64 (4.19.181-1, 4.19.194-2), python-dnspython:amd64 (1.16.0-1, 1.16.0-1+deb10u1), pve-container:amd64 (3.3-5, 3.3-6), libklibc:amd64 (2.0.6-1, 2.0.6-1+deb10u1), isc-dhcp-common:amd64 (4.4.1-2, 4.4.1-2+deb10u1), proxmox-backup-client:amd64 (1.1.8-1, 1.1.10-1), pve-manager:amd64 (6.4-8, 6.4-13), nmap:amd64 (7.70+dfsg1-6+deb10u1, 7.70+dfsg1-6+deb10u2), libgcrypt20:amd64 (1.8.4-5, 1.8.4-5+deb10u1), proxmox-backup-restore-image:amd64 (0.2.2, 0.2.3), libproxmox-backup-qemu0:amd64 (1.0.3-1, 1.1.0-1), libhogweed4:amd64 (3.4.1-1, 3.4.1-1+deb10u1), pve-kernel-helper:amd64 (6.4-2, 6.4-4), libnettle6:amd64 (3.4.1-1, 3.4.1-1+deb10u1), libxml2:amd64 (2.9.4+dfsg1-7+deb10u1, 2.9.4+dfsg1-7+deb10u2), libgnutls30:amd64 (3.6.7-4+deb10u6, 3.6.7-4+deb10u7), isc-dhcp-client:amd64 (4.4.1-2, 4.4.1-2+deb10u1), nmap-common:amd64 (7.70+dfsg1-6+deb10u1, 7.70+dfsg1-6+deb10u2), klibc-utils:amd64 (2.0.6-1, 2.0.6-1+deb10u1), libglib2.0-0:amd64 (2.58.3-2+deb10u2, 2.58.3-2+deb10u3), base-files:amd64 (10.3+deb10u9, 10.3+deb10u10)
End-Date: 2021-07-10  20:39:21

anything weird looking? Or is it safe to just reboot the host and start again?

 

[oguz]

anything weird looking? Or is it safe to just reboot the host and start again?

is apt still running? you can check with ps faxl, then we can see if it's stuck at some point in post installation hooks or so. check for apt and the subtree underneath and post it here.

if it's stuck at another prompt it's probably okay to interrupt it and run apt install -f or dpkg --configure -a, and then repeating the upgrade will continue the upgrade process (but there might be some edge cases where a task might be handled incorrectly, it's hard to say...)

 

[fuzzyduck]

i have successfully upgraded to 7 but now im stuck at initramfs prompt. Cant find rpool. Error 134.

I came from 6.4 so the upgrade shouldnt have caused any issues with grub i read.

Could you help me out?

Its a raid1 zfs bootdisk on 2 SSD's

se pic for details

https://ibb.co/JykfnPj

EDIT: zpool import shows both by pools. zpool on the mirror and TANK on the raid10
EDIT2: i can import rpool with
zpool import -R /rpool -N rpool and zpool list then shows it

How to continue to make this stick? Im an absolute novice on linux prompt. I live off tutorials lol.

thanks!

 

[oguz]

EDIT2: i can import rpool with
zpool import -R /rpool -N rpool and zpool list then shows it

then you should be able to boot and check the problem

my guess would be your zfs cachefile might be corrupted. you could try the following:

zpool set cachefile=/etc/zfs/zpool.cache rpool
update-initramfs -k all -u
reboot

 

[fuzzyduck]

https://ibb.co/YXx8MQV
this is after i press e at grub

 

[fuzzyduck]

thank you for responding once more. So if i import the rpool at initramfs, then how do i boot it from there? what command at the prompt? because issuing a reboot lands on init prompt once more.

EDIT:
update-initramfs -k all -u not found

 

[oguz]

manually import your pool like before and then type "exit". afterwards follow the commands on my previous post to reset cache and update initramfs

 

[fuzzyduck]

ahh now i have reset the cache but not issued the update-init command due to not found. Can i still proceed with exit?

 

[t.lamprecht]

EDIT2: i can import rpool with
zpool import -R /rpool -N rpool and zpool list then shows it

In the initramfs do the import to the root / mount point zpool import -R / -N rpool and then use exit to boot.

Once booted finish the upgrade, ideally with

apt update
apt full-upgrade

Trigger also an update-grub afterwards, just to be sure.

 

[fuzzyduck]

thank you. im not stuck inbetween oguz his commands. I issued a new cache file but i cannot enter the second command to update initramfs.
The command is not found.

Can i still 'exit' without updating init or will i break stuff due to that set cache file without any further action?

 

[t.lamprecht]

If you're still in the initramfs prompt then forget updating anything from the initramfs, rather reboot to get into a clean state (as we cannot know what commands you used and which state the system is in), you will still get to the initramfs prompt.

There follow my post. It basically mounts the actual root manually and then with exit we continue to boot to the full system there.

 

[fuzzyduck]

i did what you said, just imported the pool and exit.
now i am somewhere else but cant see. My monitor all of a sudden doesnt support the input timing given from the vga. biggg *SIGH*

So it didnt reboot or anything, i just dont know where im at! IP is not pingable, nor able to SSH into

EDIT: where ever i was a cntrl alt del got me back on the init prompt. Time to get an ye old monitor from somewhere

 

[fuzzyduck]

another older monitor goes into powersave as well. So im not sure that import pool and exit does the trick?

any other ideas? Like that 5sec wait timer in /etc/default/zfs?

i have backups from but oh boy this sucks bigtime.

 

[fuzzyduck]

my g*d!
I freshly installed 7 and after the grub i STILL get a blank screen! must be some new kernel thing going?
so exit after importing rpool was might be working after all

 

[need2gcm]

What is this even running on?

 

[fuzzyduck]

Intel serverboard S2400SC with onboard vga. But this topic is starting to crossover to:
https://forum.proxmox.com/threads/unknown-video-mode-on-fresh-proxmox-7-install.92623/