Proxmox 4.4-24 to 5.x upgrade gone wrong...

[Will Dennis]

Hi all,

Tried to follow the instructions at https://pve.proxmox.com/wiki/Upgrade_from_4.x_to_5.0 to upgrade the first node of my 3-node cluster from 4.4-24 to version 5.x. This seems to have gone badly wrong, because when I finished and rebooted, no pve* commands are available, and no pve* processes or ceph processes are running on the node. Took a look at output of dpkg -l, and the following packages are in status "rc":

root@proxmox-node-01:~# dpkg -l | grep ^r
rc  libboost-program-options1.55.0:amd64 1.55.0+dfsg-3                     amd64        program options library for C++
rc  libboost-regex1.55.0:amd64           1.55.0+dfsg-3                     amd64        regular expression library for C++
rc  libcwidget3:amd64                    0.5.17-2                          amd64        high-level terminal interface library for C++ (runtime files)
rc  libfcgi0ldbl                         2.4.0-8.3                         amd64        Shared library of FastCGI
rc  libleveldb1:amd64                    1.17-1                            amd64        fast key-value storage library
rc  libpython3.4-minimal:amd64           3.4.2-1+deb8u7                    amd64        Minimal subset of the Python language (version 3.4)
rc  libsigc++-2.0-0c2a:amd64             2.4.0-1                           amd64        type-safe Signal Framework for C++ - runtime
rc  libsnappy1                           1.1.2-3                           amd64        fast compression/decompression library
rc  perl-modules                         5.20.2-3+deb8u12                  all          Core Perl modules
rc  proxmox-ve                           4.4-112                           all          The Proxmox Virtual Environment
rc  pve-cluster                          4.0-55                            amd64        Cluster Infrastructure for Proxmox Virtual Environment
rc  pve-firewall                         2.0-33                            amd64        Proxmox VE Firewall
rc  pve-ha-manager                       1.0-41                            amd64        Proxmox VE HA Manager
rc  pve-manager                          4.4-24                            amd64        The Proxmox Virtual Environment
rc  python3.4                            3.4.2-1+deb8u7                    amd64        Interactive high-level object-oriented language (version 3.4)
rc  python3.4-minimal                    3.4.2-1+deb8u7                    amd64        Minimal subset of the Python language (version 3.4)
rc  qemu-server                          4.0-115                           amd64        Qemu Server Tools

(all other packages are in the "ii" state.)

Any way to recover this node and get the upgrade done, or am I beat?

 

[aaron]

Most likely due to a repository misconfiguration. What is the output if you run apt update? In later versions we have a hook preventing the full removal of Proxmox VE but I am not sure if it was already present in PVE 4.

 

[Will Dennis]

So sorry for the late reply -- please see output below:

root@proxmox-node-01:~# apt update
Get:1 http://security.debian.org stretch/updates InRelease [53.0 kB]
Ign:2 http://ftp.us.debian.org/debian stretch InRelease
Hit:3 http://ftp.us.debian.org/debian stretch Release
Hit:4 http://download.proxmox.com/debian/ceph-luminous stretch InRelease
Get:5 http://security.debian.org stretch/updates/main amd64 Packages [707 kB]
Hit:6 http://download.proxmox.com/debian stretch InRelease
Get:7 http://security.debian.org stretch/updates/main Translation-en [325 kB]
Fetched 1,084 kB in 0s (1,726 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
All packages are up to date.

 

[Will Dennis]

Also, if helpful:

root@proxmox-node-01:~# grep -R http /etc/apt/*
/etc/apt/sources.list:deb http://ftp.us.debian.org/debian stretch main contrib
/etc/apt/sources.list:deb http://security.debian.org stretch/updates main contrib
/etc/apt/sources.list.d/pve-no-subscription.list:deb http://download.proxmox.com/debian stretch pve-no-subscription
/etc/apt/sources.list.d/pve-enterprise.list:#deb https://enterprise.proxmox.com/debian stretch pve-enterprise
/etc/apt/sources.list.d/ceph.list:deb http://download.proxmox.com/debian/ceph-luminous stretch main
Binary file /etc/apt/trusted.gpg.d/debian-archive-stretch-automatic.gpg matches
Binary file /etc/apt/trusted.gpg.d/debian-archive-jessie-security-automatic.gpg matches
Binary file /etc/apt/trusted.gpg.d/debian-archive-buster-security-automatic.gpg matches
Binary file /etc/apt/trusted.gpg.d/debian-archive-stretch-security-automatic.gpg matches
Binary file /etc/apt/trusted.gpg.d/debian-archive-buster-automatic.gpg matches
Binary file /etc/apt/trusted.gpg.d/debian-archive-jessie-automatic.gpg matches

 

[fiona]

Hi,
you might want to add

http://ftp.us.debian.org/debian stretch-updates main contrib

to your sources.list to get updates for Debian packages. Note that security updates are not all updates

After that, run apt update and try installing the proxmox-ve package again.

 

[Will Dennis]

Thanks, Fabian -- that did the trick. The server is back up & healthy

Now I have the other two servers in the cluster to upgrade -- what must I do to prevent the same issue on those?

 

[fiona]

It depends on what the original problem was. But you should:

1. check that the repositories are configured correctly.
2. check that apt does not try to remove the proxmox-ve package before confirming the dist-upgrade.

If apt does try to remove the package, you might need to further dig into why (apt -o Debug::pkgProblemResolver=yes dist-upgrade might be of help then).

 

[Will Dennis]

OK, the other two upgraded just as they should - so, not sure what the heck happened with the first one... But all good now. Waiting a bit to ensure things are stable & healthy before I try a 5.x->6.x upgrade.