PVE 6 to 7 Upgrade on 4 node cluster failed on one node

rickk

New Member
Jan 22, 2023
3
0
1
I performed an upgrade from PVE 6to7 on a four node cluster and all seemed to go well. After the final node was upgrade I opened the console and the first node that I had upgraded indicated it was still on version 6.4 and that no repository was identified. I ran pve6to7 and response was already on version 7. I checked that I was set to no subscription repository and that was configured correctly.

pve6to7 now results in these errors:

root@Dell-R710-2:/var/log/apt# pve6to7
Can't locate loadable object for module PVE::IPCC in @INC (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.28.1 /usr/local/share/perl/5.28.1 /usr/lib/x86_64-linux-gnu/perl5/5.28 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.28 /usr/share/perl/5.28 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/PVE/Cluster.pm line 17.
Compilation failed in require at /usr/share/perl5/PVE/Cluster.pm line 17.
BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Cluster.pm line 17.
Compilation failed in require at /usr/share/perl5/PVE/API2/APT.pm line 15.
BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2/APT.pm line 15.
Compilation failed in require at /usr/share/perl5/PVE/CLI/pve6to7.pm line 6.
BEGIN failed--compilation aborted at /usr/share/perl5/PVE/CLI/pve6to7.pm line 6.
Compilation failed in require at /usr/bin/pve6to7 line 6.
BEGIN failed--compilation aborted at /usr/bin/pve6to7 line 6.
root@Dell-R710-2:/var/log/apt# pveversion -v
Can't locate loadable object for module PVE::IPCC in @INC (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.28.1 /usr/local/share/perl/5.28.1 /usr/lib/x86_64-linux-gnu/perl5/5.28 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.28 /usr/share/perl/5.28 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/PVE/Cluster.pm line 17.
Compilation failed in require at /usr/share/perl5/PVE/Cluster.pm line 17.
BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Cluster.pm line 17.
Compilation failed in require at /usr/share/perl5/PVE/API2/APT.pm line 15.
BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2/APT.pm line 15.
Compilation failed in require at /usr/bin/pveversion line 7.
BEGIN failed--compilation aborted at /usr/bin/pveversion line 7.

tried to fix dpkg with configure -a and receive these errors
root@Dell-R710-2:/var/log/apt# dpkg --configure -a
dpkg: dependency problems prevent configuration of librados2-perl:
librados2-perl depends on perlapi-5.32.1; however:
Package perlapi-5.32.1 is not installed.

dpkg: error processing package librados2-perl (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of libpve-u2f-server-perl:
libpve-u2f-server-perl depends on perlapi-5.32.1; however:
Package perlapi-5.32.1 is not installed.

dpkg: error processing package libpve-u2f-server-perl (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of pve-cluster:
pve-cluster depends on perlapi-5.32.1; however:
Package perlapi-5.32.1 is not installed.

dpkg: error processing package pve-cluster (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent processing triggers for pve-manager:
pve-manager depends on librados2-perl; however:
Package librados2-perl is not configured yet.
pve-manager depends on pve-cluster (>= 6.0-4); however:
Package pve-cluster is not configured yet.

dpkg: error processing package pve-manager (--configure):
dependency problems - leaving triggers unprocessed
dpkg: dependency problems prevent processing triggers for pve-ha-manager:
pve-ha-manager depends on pve-cluster (>= 3.0-17); however:
Package pve-cluster is not configured yet.

dpkg: error processing package pve-ha-manager (--configure):
dependency problems - leaving triggers unprocessed
Errors were encountered while processing:
librados2-perl
libpve-u2f-server-perl
pve-cluster
pve-manager
pve-ha-manager


Here is last bit of apt history log (this was not my initial attempt at upgrade - I tried again to see if that would fix my issue)

Start-Date: 2023-01-21 17:36:50
Commandline: apt dist-upgrade
Upgrade: pve-cluster:amd64 (6.4-1, 7.3-2), librados2-perl:amd64 (1.1-2, 1.3-1), libpve-u2f-server-perl:amd64 (1.1-1, 1.1-2)
Error: Sub-process /usr/bin/dpkg returned an error code (1)
End-Date: 2023-01-21 17:36:54

Here are today's syslog errors associated with PVE

Jan 22 04:13:21 Dell-R710-2 pveupdate[4468]: Can't locate loadable object for module PVE::IPCC i
n @INC (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.28.1 /usr/local/share/pe
rl/5.28.1 /usr/lib/x86_64-linux-gnu/perl5/5.28 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5
.28 /usr/share/perl/5.28 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/s
hare/perl5/PVE/Cluster.pm line 17.
Jan 22 04:13:21 Dell-R710-2 pveupdate[4468]: Compilation failed in require at /usr/share/perl5/P
VE/Cluster.pm line 17.
Jan 22 04:13:21 Dell-R710-2 pveupdate[4468]: BEGIN failed--compilation aborted at /usr/share/per
l5/PVE/Cluster.pm line 17.
Jan 22 04:13:21 Dell-R710-2 pveupdate[4468]: Compilation failed in require at /usr/share/perl5/P
VE/API2/ACMEPlugin.pm line 12.
Jan 22 04:13:21 Dell-R710-2 pveupdate[4468]: BEGIN failed--compilation aborted at /usr/share/per
l5/PVE/API2/ACMEPlugin.pm line 12.
Jan 22 04:13:21 Dell-R710-2 pveupdate[4468]: Compilation failed in require at /usr/share/perl5/P
VE/NodeConfig.pm line 11.
Jan 22 04:13:21 Dell-R710-2 pveupdate[4468]: BEGIN failed--compilation aborted at /usr/share/per
l5/PVE/NodeConfig.pm line 11.
Jan 22 04:13:21 Dell-R710-2 pveupdate[4468]: Compilation failed in require at /usr/bin/pveupdate
line 12.
Jan 22 04:13:21 Dell-R710-2 pveupdate[4468]: BEGIN failed--compilation aborted at /usr/bin/pveup
date line 12.
Jan 22 04:13:21 Dell-R710-2 systemd[1]: pve-daily-update.service: Main process exited, code=exit
ed, status=2/INVALIDARGUMENT
Jan 22 04:13:21 Dell-R710-2 systemd[1]: pve-daily-update.service: Failed with result 'exit-code'
.
Jan 22 04:13:21 Dell-R710-2 systemd[1]: Failed to start Daily PVE download activities.
Jan 22 04:17:01 Dell-R710-2 CRON[4492]: (root) CMD ( cd / && run-parts --report /etc/cron.hour
ly)

Here is my sources.list

# deb http://ftp.debian.org/debian bullseye main contrib
# deb http://ftp.debian.org/debian bullseye-updates main contrib

# PVE pve-no-subscription repository provided by proxmox.com,
# NOT recommended for production use
deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription

# security updates
deb http://security.debian.org/debian-security bullseye-security main contrib
 
adding output of dpkg --audit

root@Dell-R710-2:/etc/apt# dpkg --audit
The following packages have been unpacked but not yet configured.
They must be configured using dpkg --configure or the configure
menu option in dselect for them to work:
libpve-u2f-server-perl Perl bindings for libu2f-server
librados2-perl Perl bindings for librados
pve-cluster "pmxcfs" distributed cluster filesystem for Proxmox Virtu

The following packages have been triggered, but the trigger processing
has not yet been done. Trigger processing can be requested using
dselect or dpkg --configure --pending (or dpkg --triggers-only):
pve-ha-manager Proxmox VE HA Manager
pve-manager Proxmox Virtual Environment Management Tools
 
I was able to fix this -

While I wasn't using ceph, I did update the ceph repository to point to Bullseye. In sources.list I had commented out the first two lines because I thought that was necessary when using no subscription. I uncommented those lines, ran the fix install command that rant apt update and apt upgrade again. Rebooted the node and it came up green. But still said no repository on the summary page in the gui. Clicked on "updates" and there we several to install. Did upgrade from gui and that seems to have fixed everything. Node is up and points to no subscription repository.

deb http://ftp.debian.org/debian bullseye main contrib
deb http://ftp.debian.org/debian bullseye-updates main contrib

# PVE pve-no-subscription repository provided by proxmox.com,
# NOT recommended for production use
deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription

# security updates
deb http://security.debian.org/debian-security bullseye-security main contrib
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!