6 node cluster showing different version of Proxmox virtual environment GUI even after upgrade

Apr 16, 2021
4
0
1
35
Hi All,

We have a 6 node Proxmox cluster which was upgraded first from version 5 to 6.1. After receiving recent upgrades like we have upgraded all our 6 nodes to version 6.3 successfully. But after carefully checking all the nodes, we have noticed that, 3 nodes out of 6 are showing Proxmox Virtual Environment 6.1-7 (GUI) and other 3 nodes showing Virtual Environment 6.3-3 (GUI).
All the nodes were upgraded using -

apt update
apt dist-upgrade

We have even tried apt dist-upgrade again on the affected 3 nodes but with no success. We have also noticed that pve-manager package on these affected nodes is showing - pve-manager: 6.1-7 (running version: 6.1-7/13e58d5e)

and on the nodes which are correct is showing - pve-manager: 6.3-3 (running version: 6.3-3/eee5f901).

Node1 (Affected node) package summary -

proxmox-ve: 6.3-1 (running kernel: 5.4.106-1-pve)
pve-manager: 6.1-7 (running version: 6.1-7/13e58d5e)
pve-kernel-5.4: 6.3-8
pve-kernel-helper: 6.3-8
pve-kernel-5.3: 6.1-6
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
pve-kernel-4.15: 5.4-12
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.13-3-pve: 5.3.13-3
pve-kernel-4.15.18-24-pve: 4.15.18-52
pve-kernel-4.10.17-2-pve: 4.10.17-20
ceph: 12.2.13-pve1~bpo9
ceph-fuse: 12.2.13-pve1~bpo9
corosync: 3.1.0-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libpve-access-control: 6.0-6
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.0-11
libpve-guest-common-perl: 3.0-3
libpve-http-server-perl: 3.0-4
libpve-storage-perl: 6.1-4
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 3.2.1-1
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.4-9
pve-cluster: 6.2-1
pve-container: 3.0-19
pve-docs: 6.3-1
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.2-2
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.1.0-8
pve-xtermjs: 4.7.0-3
qemu-server: 6.1-5
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.4-pve1



Node6 (Correct Node) Package Version :

proxmox-ve: 6.3-1 (running kernel: 5.4.78-2-pve)
pve-manager: 6.3-3 (running version: 6.3-3/eee5f901)
pve-kernel-5.4: 6.3-3
pve-kernel-helper: 6.3-3
pve-kernel-5.4.78-2-pve: 5.4.78-2
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph: 14.2.16-pve1
ceph-fuse: 14.2.16-pve1
corosync: 3.1.0-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.0.7
libproxmox-backup-qemu0: 1.0.2-1
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.3-3
libpve-guest-common-perl: 3.1-4
libpve-http-server-perl: 3.1-1
libpve-storage-perl: 6.3-6
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.0.8-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.4-5
pve-cluster: 6.2-1
pve-container: 3.3-3
pve-docs: 6.3-1
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.1-3
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.1.0-8
pve-xtermjs: 4.7.0-3
qemu-server: 6.3-5
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 0.8.5-pve1


Also I have attached screenshot of the Proxmox Virtual environment GUI.
 

Attachments

  • Proxmox_Node1.JPG
    Proxmox_Node1.JPG
    15.8 KB · Views: 8
  • Proxmox_node6.JPG
    Proxmox_node6.JPG
    24.2 KB · Views: 8
have you compared the repositories for your nodes if are the same?
did you reboot after upgrade to 6.3?
 
Thank you for your reply Moayad.

Yes we have done reboot after upgrade. We have compared the packages and yes some differences are there in the version. But even after doing apt update & apt dist-upgrade they are not getting upgraded. If you refer above, I have given the package versions from 2 nodes.
I tried running update again below was the error that I got -

root@srv01:~# apt dist-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
geoip-database-extra libegl1-mesa libfdt1 libgl1-mesa-glx libglew2.0 libjs-openlayers liblttng-ust-ctl2 libpcre16-3 libqgsttools-p1 libtxc-dxtn-s2tc
libwayland-egl1-mesa libwireshark8 libwiretap6 libwscodecs1 libwsutil7 pve-kernel-4.15 pve-kernel-4.15.18-24-pve pve-kernel-5.3.13-3-pve python-pyasn1
Use 'apt autoremove' to remove them.
The following packages have been kept back:
libpve-access-control libpve-common-perl libpve-guest-common-perl libpve-http-server-perl libpve-storage-perl lxc-pve pve-container pve-manager
qemu-server
0 upgraded, 0 newly installed, 0 to remove and 9 not upgraded.
root@srv01:~# apt full-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
geoip-database-extra libegl1-mesa libfdt1 libgl1-mesa-glx libglew2.0 libjs-openlayers liblttng-ust-ctl2 libpcre16-3 libqgsttools-p1 libtxc-dxtn-s2tc
libwayland-egl1-mesa libwireshark8 libwiretap6 libwscodecs1 libwsutil7 pve-kernel-4.15 pve-kernel-4.15.18-24-pve pve-kernel-5.3.13-3-pve python-pyasn1
Use 'apt autoremove' to remove them.
The following packages have been kept back:
libpve-access-control libpve-common-perl libpve-guest-common-perl libpve-http-server-perl libpve-storage-perl lxc-pve pve-container pve-manager




Regards,
Aniruddha
 
Hi Moayad,

Below is the output from two nodes -

Node1(Affected & dont have ceph) -

root@fsrv01:~# grep '' /etc/apt/sources.list.d/*
/etc/apt/sources.list.d/ceph.list:deb http://download.proxmox.com/debian/ceph-luminous stretch main
/etc/apt/sources.list.d/pve-enterprise.list:deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise


Node6(Good node & Has CEPH) -

root@srv06:~# grep '' /etc/apt/sources.list.d/*
/etc/apt/sources.list.d/ceph.list:deb http://download.proxmox.com/debian/ceph-nautilus buster main
/etc/apt/sources.list.d/pve-enterprise.list:deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise


Regards,
Aniruddha
 
Hi Moayad,

Please find below the output -


root@srv01:~# apt update
Get:1 http://security.debian.org buster/updates InRelease [65.4 kB]
Hit:2 http://download.proxmox.com/debian/ceph-luminous stretch InRelease
Hit:3 http://ftp.de.debian.org/debian buster InRelease
Get:4 http://ftp.de.debian.org/debian buster-updates InRelease [51.9 kB]
Hit:5 https://enterprise.proxmox.com/debian/pve buster InRelease
Fetched 117 kB in 1s (135 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
12 packages can be upgraded. Run 'apt list --upgradable' to see them.

Additional incase you want to see the packages -

root@srv01:~# apt list --upgradable
Listing... Done
libproxmox-backup-qemu0/stable 1.0.3-1 amd64 [upgradable from: 1.0.2-1]
libpve-access-control/stable 6.1-3 all [upgradable from: 6.0-6]
libpve-common-perl/stable 6.3-5 all [upgradable from: 6.0-11]
libpve-guest-common-perl/stable 3.1-5 all [upgradable from: 3.0-3]
libpve-http-server-perl/stable 3.1-1 all [upgradable from: 3.0-4]
libpve-storage-perl/stable 6.3-8 all [upgradable from: 6.1-4]
lxc-pve/stable 4.0.6-2 amd64 [upgradable from: 3.2.1-1]
pve-container/stable 3.3-4 all [upgradable from: 3.0-19]
pve-i18n/stable 2.3-1 all [upgradable from: 2.2-2]
pve-manager/stable 6.3-6 amd64 [upgradable from: 6.1-7]
pve-qemu-kvm/stable 5.2.0-5 amd64 [upgradable from: 5.1.0-8]
qemu-server/stable 6.3-10 amd64 [upgradable from: 6.1-5]


Thank you again for your quick replies. :)

Regards,
Aniruddha
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!