proxmox update broke

jprcrna · Nov 22, 2023

Hello,

One proxmox server was updating when we have a break of the internet connexion.

Since, the debian tell me to do a "dpkg --configure -a"
The command say : "setting up pve-manage (8.0.9)" and run for 15 hours et nothing change.

We try to stop it (ctrl+c) and the server say :
"dpkg: error processing package pve-manager (--configure):
installed pve-manager package post-installation script subprocess was interrupted
dpkg: dependendy problems prevent configuration of proxmox-ve:
proxmox-ve depends on pve-manager (>= 8.0.4); however:
package pve-manager is not configured yet

dpkg: error processing package pve-ve(--configure):
setting up proxmox-kernel-6.2 (6.2.16-19)
setting up pve-kernel-6.2 (8.0.5)
processing triggers for man-bd (2.11.2-2)
errors were encountered while processing:
pve-manage
proxmox-ve"

what can i do to correct this problem please ?!

P.S. : i haven't find the solution on the forum
P.S. 2 : it's a new server, no vm on it, we can do all things on it

LnxBil · Nov 22, 2023

Does

Code:

apt install -f

yields anything?

LPT: always use screen or tmux on the server to be able to continue your work. I also had those "network quicks" and lost work.

Moayad · Nov 22, 2023

Hello,

May you try first to clean and -f install and then do `dpkg --configure -a`:

Code:

apt autoclean
apt install -f

If the dpkg --confgure -a stuck again please provide us with the output of following command:

Code:

dpkg --get-selections | grep hold

jprcrna · Nov 22, 2023

Thank you LnxBil and Moayad, i try now and come back to you

it seems to be block in the apt install -f and pve-manager

l wait for 15mn ?!

jprcrna · Nov 22, 2023

ok i'm back
no news, so i must reset i can't ctrl+c this time (stuck after apt install -f)

dpkg --get-selections | grep hold
return noting

edit :
i retry dpkg --configure -a => stuck
so dpkg --get-selections | grep hold
nothing

LnxBil · Nov 22, 2023

Any information in any logfile in /var/log?

Maybe strace the dpkg call.

jprcrna · Nov 23, 2023

Hello this morning !

"Maybe strace the dpkg call."
i don't know how to do, i'm searching

/var/log
dpkg.log :
2023-11-22 13:09:05 configure pve-kernel-6.2:all 8.0.5 <none>
2023-11-22 13:09:05 status unpacked pve-kernel-6.2:all 8.0.5
2023-11-22 13:09:05 status half-configured pve-kernel-6.2:all 8.0.5
2023-11-22 13:09:05 status installed pve-kernel-6.2:all 8.0.5
2023-11-22 13:09:05 trigproc man-db:amd64 2.11.2-2 <none>
2023-11-22 13:09:05 status half-configured man-db:amd64 2.11.2-2
2023-11-22 13:09:06 status installed man-db:amd64 2.11.2-2
2023-11-22 13:55:38 startup packages configure
2023-11-22 13:55:38 configure pve-manager:amd64 8.0.9 <none>
2023-11-22 13:55:38 status half-configured pve-manager:amd64 8.0.9
2023-11-22 16:16:03 startup packages configure
2023-11-22 16:16:03 configure pve-manager:amd64 8.0.9 <none>
2023-11-22 16:16:03 status half-configured pve-manager:amd64 8.0.9

this is all the time this messages

nothing special in pveam.log

LnxBil · Nov 23, 2023

LnxBil said:
Maybe strace the dpkg call.

You may run with apt:

Code:

strace --follow-forks --output=/tmp/strace.out apt install -f

Let it run a couple of minutes and then abort with CTRL+C. Look at the file in /tmp/strace.out at the end for strange behavior and/or upload the file to pastebin et. al.

(If you get strace: command not found, we may have a problem, because you cannot install new packages with apt: hen-egg-problem).

jprcrna · Nov 23, 2023

ok it's done
trace.out 6.5mo !!

ok here the log
https://pastebin.com/xiqSXXBB
i cut it cause it's too big for pastebin

thank you for your help !

sb-jw · Nov 23, 2023

Just paste it here in the forum instead of on external services.

jprcrna · Nov 23, 2023

Code:

165783 prctl(PR_CAPBSET_READ, CAP_CHOWN) = 1
165783 prctl(PR_GET_SECUREBITS)         = 0
165783 prctl(PR_GET_NO_NEW_PRIVS, 0, 0, 0, 0) = 0
165783 prctl(PR_CAP_AMBIENT, PR_CAP_AMBIENT_IS_SET, CAP_CHOWN, 0, 0) = 0
165783 getrandom("\x4d\xd2\x9f\x47\x19\xf9\x63\x3f", 8, GRND_NONBLOCK) = 8
165783 brk(NULL)                        = 0x559b08b23000
165783 brk(0x559b08b44000)              = 0x559b08b44000
165783 statfs("/sys/fs/selinux", 0x7ffd02c1ffe0) = -1 ENOENT (No such file or directory)
165783 statfs("/selinux", 0x7ffd02c1ffe0) = -1 ENOENT (No such file or directory)
165783 openat(AT_FDCWD, "/proc/filesystems", O_RDONLY|O_CLOEXEC) = 3
165783 newfstatat(3, "", {st_mode=S_IFREG|0444, st_size=0, ...}, AT_EMPTY_PATH) = 0
165783 read(3, "nodev\tsysfs\nnodev\ttmpfs\nnodev\tbd"..., 1024) = 452
165783 read(3, "", 1024)                = 0
165783 close(3)                         = 0
165783 access("/etc/selinux/config", F_OK) = -1 ENOENT (No such file or directory)
165783 prctl(PR_CAPBSET_READ, CAP_MAC_OVERRIDE) = 1
165783 prctl(PR_CAPBSET_READ, 0x30 /* CAP_??? */) = -1 EINVAL (Invalid argument)
165783 prctl(PR_CAPBSET_READ, CAP_CHECKPOINT_RESTORE) = 1
165783 prctl(PR_CAPBSET_READ, 0x2c /* CAP_??? */) = -1 EINVAL (Invalid argument)
165783 prctl(PR_CAPBSET_READ, 0x2a /* CAP_??? */) = -1 EINVAL (Invalid argument)
165783 prctl(PR_CAPBSET_READ, 0x29 /* CAP_??? */) = -1 EINVAL (Invalid argument)
165783 getpid()                         = 165783
165783 umask(022)                       = 022
165783 openat(AT_FDCWD, "/proc/self/stat", O_RDONLY|O_CLOEXEC) = 3
165783 newfstatat(3, "", {st_mode=S_IFREG|0444, st_size=0, ...}, AT_EMPTY_PATH) = 0
165783 read(3, "165783 (systemd-tty-ask) R 16578"..., 1024) = 330
165783 ioctl(3, TCGETS, 0x7ffd02c1f920) = -1 ENOTTY (Inappropriate ioctl for device)
165783 read(3, "", 1024)                = 0
165783 close(3)                         = 0
165783 newfstatat(AT_FDCWD, "/run/systemd/ask-password-block", {st_mode=S_IFDIR|0700, st_size=80, ...}, 0) = 0
165783 mknodat(AT_FDCWD, "/run/systemd/ask-password-block/136:0", S_IFIFO|0600) = -1 EEXIST (File exists)
165783 openat(AT_FDCWD, "/run/systemd/ask-password-block/136:0", O_RDONLY|O_NOCTTY|O_NONBLOCK|O_CLOEXEC) = 3
165783 newfstatat(AT_FDCWD, "/run/systemd", {st_mode=S_IFDIR|0755, st_size=560, ...}, 0) = 0
165783 mkdirat(AT_FDCWD, "/run/systemd/ask-password", 0755) = -1 EEXIST (File exists)
165783 newfstatat(AT_FDCWD, "/run/systemd/ask-password", {st_mode=S_IFDIR|0755, st_size=40, ...}, 0) = 0
165783 rt_sigprocmask(SIG_SETMASK, [TERM], NULL, 8) = 0
165783 signalfd4(-1, [TERM], 8, SFD_CLOEXEC|SFD_NONBLOCK) = 4
165783 inotify_init1(IN_CLOEXEC)        = 5
165783 inotify_add_watch(5, "/run/systemd/ask-password", IN_CLOSE_WRITE|IN_MOVED_TO) = 1
165783 openat(AT_FDCWD, "/run/systemd/ask-password", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 6
165783 newfstatat(6, "", {st_mode=S_IFDIR|0755, st_size=40, ...}, AT_EMPTY_PATH) = 0
165783 getdents64(6, 0x559b08b23c90 /* 2 entries */, 32768) = 48
165783 getdents64(6, 0x559b08b23c90 /* 0 entries */, 32768) = 0
165783 close(6)                         = 0
165783 ppoll([{fd=4, events=POLLIN}, {fd=5, events=POLLIN}], 2, NULL, NULL, 8 <unfinished ...>
165684 <... pselect6 resumed>)          = 0 (Timeout)
165684 wait4(165696, 0x7fffc51449b8, WNOHANG, NULL) = 0
165684 pselect6(25, [0 22 24], NULL, NULL, {tv_sec=0, tv_nsec=50000000}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
165684 wait4(165696, 0x7fffc51449b8, WNOHANG, NULL) = 0
165684 pselect6(25, [0 22 24], NULL, NULL, {tv_sec=0, tv_nsec=50000000}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
165684 wait4(165696, 0x7fffc51449b8, WNOHANG, NULL) = 0
165684 pselect6(25, [0 22 24], NULL, NULL, {tv_sec=0, tv_nsec=50000000}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
165684 wait4(165696, 0x7fffc51449b8, WNOHANG, NULL) = 0
165684 pselect6(25, [0 22 24], NULL, NULL, {tv_sec=0, tv_nsec=50000000}, {sigmask=[], sigsetsize=8}) = ? ERESTARTNOHAND (To be restarted if no handler)
165684 --- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
165684 rt_sigreturn({mask=[]})          = -1 EINTR (Interrupted system call)
165684 wait4(165696, 0x7fffc51449b8, WNOHANG, NULL) = 0
165684 pselect6(25, [0 22 24], NULL, NULL, {tv_sec=0, tv_nsec=50000000}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
165684 wait4(165696, 0x7fffc51449b8, WNOHANG, NULL) = 0
165684 pselect6(25, [0 22 24], NULL, NULL, {tv_sec=0, tv_nsec=50000000}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
165684 wait4(165696, 0x7fffc51449b8, WNOHANG, NULL) = 0
165684 pselect6(25, [0 22 24], NULL, NULL, {tv_sec=0, tv_nsec=50000000}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
165684 wait4(165696, 0x7fffc51449b8, WNOHANG, NULL) = 0
165684 pselect6(25, [0 22 24], NULL, NULL, {tv_sec=0, tv_nsec=50000000}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
165684 wait4(165696, 0x7fffc51449b8, WNOHANG, NULL) = 0
165684 pselect6(25, [0 22 24], NULL, NULL, {tv_sec=0, tv_nsec=50000000}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
165684 wait4(165696, 0x7fffc51449b8, WNOHANG, NULL) = 0
165684 pselect6(25, [0 22 24], NULL, NULL, {tv_sec=0, tv_nsec=50000000}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
165684 wait4(165696, 0x7fffc51449b8, WNOHANG, NULL) = 0

sb-jw · Nov 23, 2023

There is also a function for attaching files. My statement was not copy the contents of Pastebin into the forum. Rather, my statement was to attach the file as a whole here in the forum so that all the information is available to us.

jprcrna · Nov 23, 2023

oh ok, i can do that, i understand

edit : for doing it, i return on the proxmox and the file grow again (30mo)
so i go to stop on the command and i have a progress running this time !!
i make a screenshot and i'm waiting...

LnxBil · Nov 23, 2023

While the apt install is running, can you please post the output of ps auxf in code tags?

jprcrna · Nov 23, 2023

LnxBil said:
While the apt install is running, can you please post the output of ps auxf in code tags?

I don't understand what to do, i'm sorry

Always 20%, 2 hours and 0% of progress

LnxBil · Nov 23, 2023

Open another terminal and post the ouput of ps auxf in CODE tags here on the forums.

jprcrna · Nov 23, 2023

ok thanks

LnxBil · Nov 23, 2023

Now we know what's going on (as the strace already implied):

Code:

root        4077  0.0  0.0   6124  3792 tty1     Ss   Nov22   0:00 /bin/login -p --
root        6248  0.0  0.0   8372  5120 tty1     S    Nov22   0:00  \_ -bash
root      165681  0.3  0.0   8020  3840 tty1     S+   09:28   0:50      \_ strace --follow-forks --output=/tmp/trace.out apt install -f
root      165684  0.1  0.0  72272 61092 tty1     S+   09:28   0:25          \_ apt install -f
root      165696  0.0  0.0  11204  4920 pts/0    Ss+  09:28   0:00              \_ /usr/bin/dpkg --status-fd 25 --configure --pending
root      165697  0.0  0.0   2576  1536 pts/0    S+   09:28   0:00                  \_ /bin/sh /var/lib/dpkg/info/pve-manager.postinst configure 8.0.3
root      165782  0.0  0.0  16424  5632 pts/0    S+   09:28   0:00                      \_ systemctl reload-or-restart pvescheduler.service
root      165783  0.0  0.0  16296  6144 pts/0    S+   09:28   0:00                          \_ /bin/systemd-tty-ask-password-agent --watch

It waits for some kind of password input in the background. Can you try to send a HUP signal to the password input process?

Code:

kill -HUP 165783

If that does not help, maybe @Moayad or any other Proxmox staff member can jump in.

jprcrna · Nov 23, 2023

Nice LnxBil, and thank you

Can you try to send a HUP signal to the password input process?

how can i do to send that ?

LnxBil · Nov 23, 2023

jprcrna said:
how can i do to send that ?

I gave you the actual command to do that

proxmox update broke

Member

Distinguished Member

Proxmox Staff Member

Member

Member

Distinguished Member

Member

Distinguished Member

Member

Famous Member

Member

Famous Member

Member

Distinguished Member

Member

Distinguished Member

Member

Attachments

Distinguished Member

Member

Distinguished Member

We value your privacy