Hi, I have decided to just migrate everything to a fresh cluster by moving nodes one-by one, and migrate VMs via Datacenter manager.
This cluster has been alive, upgraded, with nodes added, removed since early PVE7 - so it's high noon to start...
I have a 6-node [pve2 ... pve9] PVE cluster [base versions: pvemgr 9.1.4, ceph 19.2.3] and I have started updating the nodes. After pve6 node upgrade two small OSDs failed to start back up, I destroyed them [as they would have been removed soon...