[SOLVED] VM stops after upgrading and restarting other nodes of the cluster

[Angry Mandragora]

I have a 3 node test cluster set up to test 6.4 to 7.x upgrade.
Ceph (version 14) is installed, one OSD/one monitor on each node, a pool is configured using the OSDs.
A VM is created using the pool. It runs on node03.
I upgrade Ceph from 14 to 15 without issues, restart the other nodes, migrate the VM to one of the restarted nodes, restart the 3rd node as well, all good, Ceph is now v15.
After this I proceed to upgrading the OS, I do the same, upgrade two nodes, and I leave alone the node that runs the VM.
All goes well, until I restart the two nodes, when I lose connection to the node that I didn't even touch yet (and which is running the VM). After some time the cluster comes back, but the VM is in stopped state, starting after some time.
Am I missing something or is it impossible to do a major release update without downtime?

 

[fiona]

Hi,
did you restart two of the nodes at the same time? Or maybe not wait until all (Ceph) services were fully up? With 3 nodes, you can only restart one node at a time or the cluster will loose it's quorum and Ceph also can't continue to provide service.

 

[Angry Mandragora]

Indeed I did! I feel silly now. Thank you for the super quick answer!