PVE 7 to PVE 8 Upgraded and Installed from PVE 8 iso inconsistencies

[TimRyan]

I have PVE 8.0.4 Cluster with 8 Dell rack servers. 5 that were originally installed with an early release of PVE 7.2 built from PVE Debian Bullseye iso and later upgraded multiple times to 8.0.2 and later 8.0.4. on Debian Bookworm. Recently I added 3 new servers installed from scratch on the same hardware type, Dell r710 servers. These were new installations from the latest releases of PVE 8.0.2 with Debian Bullseye from iso's and upgraded to the latest updates as are all of the servers in the cluster

There are differences in the components and some anomalies in behavior. The irritating failure is as follows;

Nodes 1 thru 5 were earlier and upgraded installs and node 1 was the original cluster quorum starter. The first 5 nodes function without issues. The new nodes 6, 7, and 8 were installed from the latest iso's with Bullseye from scratch and added to the cluster without issue but,

The three latest nodes refuse to allow the VM's hosted to connect to the VNC functions. I can access them from external ssh sessions but not from the PVE WebGui interface's shell screens. I have run the pveversion -v shell commands at both prox-1 and prox-6 for comparison and there is little doubt that there are differences, but which is the cause of the VNC failures is a mystery to me. Could any on here give me a pointer as to what the issues is cause by?

The notable and obvious difference is the kernel

Nodes prox-1 thru prox-5

proxmox-ve: 8.0.2 (running kernel: 6.2.16-15-pve)
pve-manager: 8.0.4 (running version: 8.0.4/d258a813cfa6b390)

Nodes prox-6 thru prox-8

proxmox-ve: 8.0.2 (running kernel: 6.2.16-19-pve)
pve-manager: 8.0.4 (running version: 8.0.4/d258a813cfa6b390)

 

[Moayad]

Hi,

Are the repositories the same on both nodes?

grep '' /etc/apt/sources.list && grep '' /etc/apt/sources.list.d

 

[TimRyan]

They were not, and in fact they were a mess as they had been constructed and added to the cluster over a period of months and not properly examined for discrepancies and consistencies. Thanks for the perspective, The very latest three servers were installed directly from the pve 8.0.2 iso the others had been added and upgraded from earlier pve 6 and 7.x instances. I did a systematic review of

/etc/apt# sources.list
and
/etc/apt/sources.list.d# ceph.list and pve-enterprise.list

This leads me to ask the question, there appears to be duplication in these three files, and what is the best practices and rationale behind these structures ? There are multiple conflicting recommendations for how these can or should be configured particularly around the issues of subscriptions or no subscriptions depending on where the information came from

This is what I have today and has resolved all of the operational issues to date;

The default /etc/apt/sources.list file contains these two lines;

deb http://ftp.ca.debian.org/debian bookworm main contrib

deb http://ftp.ca.debian.org/debian bookworm-updates main contrib

The /etc/apt/sources.list.d# contains these two files as edited;

pve-enterprise.list contains these two lines
# deb https://enterprise.proxmox.com/debian/pve bookworm pve-enterprise
deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription

ceph.list contains these two lines
#deb https://enterprise.proxmox.com/debian/ceph-quincy bookworm enterprise
deb http://download.proxmox.com/debian/ceph-quincy bookworm no-subscription

This is all functional now but in a clustered environment it appears to be critical that there are no inconsistencies between nodes in a cluster, and that updates are carried out systematically in the monitored host environment.

 

[Neobin]

The default /etc/apt/sources.list file contains these two lines;

deb http://ftp.ca.debian.org/debian bookworm main contrib

deb http://ftp.ca.debian.org/debian bookworm-updates main contrib

The Debian security repository is missing:
https://pve.proxmox.com/wiki/Package_Repositories -> "Sources.list"