New PVE 9 node shows "unknown"/offline in GUI from PVE 7/8 nodes, despite cluster status/quorum being OK

iwik

Active Member
Oct 7, 2021
95
21
28
44
Hi all,

I added a new node to my existing PVE cluster ( mixed versions — most nodes are 8.x, one is still 7.x, and the new node is 9.2.2).

Context: the node name "pve8" was previously used by a different physical node, which I properly removed from the cluster via `pvecm delnode` some time ago. I've now reused that name for a new node, installed fresh with PVE 9.2.2, with a new/different IP address.

The join itself completed cleanly via `pvecm add`:

waiting for quorum...OK
(re)generate node files
generate new node certificate
merge authorized SSH keys
generated new node certificate, restart pveproxy and pvedaemon services
successfully added node 'pve8' to cluster.
TASK OK

After joining, cluster status looks healthy from every node I check:

- `pvecm status` — quorate, correct number of votes
- `corosync-cfgtool -s` — no faults on links
- `/etc/pve/.members` — new node listed with `"online": 1` and correct (new) IP, consistent across all nodes I checked

However, in the web GUI, the new node shows as offline/grey/"unknown" when viewed from the older nodes (the 7.x node, and the 8.x nodes). Viewed from the new node itself (9.2.2), everything looks fine — online, normal CPU/RAM/uptime stats.

So corosync/pmxcfs membership looks fully healthy, but something above that layer isn't propagating/rendering correctly to the older nodes' GUI.

Already checked/ruled out:
- Quorum/corosync connectivity — fine (see above)
- Time sync between nodes — no significant drift
- `pvestatd` on the new node — active, no errors in journal

Questions:
- Is this a known limitation when an older PVE GUI (7.x/8.x) tries to display a node running a newer major version (9.x) — i.e. it can't parse the newer status broadcast format?
- Could reusing a previously-removed node name (with a new IP) leave any stale reference behind that would cause this specifically, versus just adding a brand-new node name?


Thanks for any input!
 
The combination of 7.x with 9.x is untested and explicitly unsupported. Are the PVE 8 nodes updated to the latest 8.4.y? Otherwise it is untested and could be unsupported. PVE 8.x is (about to be) unsupported completely. I'm afraid your cluster is rather late to the party and the technical staff might already gone home.
Make sure to update all your nodes to the latest 8.4.y to support mixing them with PVE 9. Or start a fresh new PVE 9 cluster (by recycling old nodes) and test/move/migrate the VMs over from backups or using Proxmox Datacenter Manager.
However, it might not be safe to reuse the same name. Maybe upgrade the node instead if you want to reuse the name and/or IP. I'm not sure, sorry.