dist-upgrade: Running VMs & CEPH cluster: Can I restart these daemons on my 3-node cluster

[matt.]

...node after node, without that "breaking" running VMs?

PVE 8.1.4
CEPH 3-node cluster

Any help is much appreciated.

 

[sb-jw]

What exactly is your question now? Whether you should confirm that or not? Ultimately it doesn't matter, you should reboot the node afterwards anyway.

 

[matt.]

What exactly is your question now?

The question if I can restart the services without running VMs being affected.

Ultimately it doesn't matter, you should reboot the node afterwards anyway.

Why should I reboot the node? Linux isn't Windows. Usually it should be enough to restart the service

 

[sb-jw]

The question if I can restart the services without running VMs being affected.

During an update or upgrade there should be no VMs on it and your CEPH should at best be in noout.

Why should I reboot the node? Linux isn't Windows. Usually it should be enough to restart the service

Long uptimes are not a status symbol in Linux either and simply have nothing to do with the fact that it is not Windows. This statement alone is complete nonsense and shows that the security of the systems is not important.

You can't easily put new kernels into operation live and some services may still be stuck on old paths to libraries or you may not have caught all the services.

 

[UdoB]

During an update or upgrade there should be no VMs on it

For a "real" dist-upgrade (e.g. PVE 7.x to 8.0) you are definitely right. But for a weekly, normal upgrade (executed via "apt dist-upgrade" of course) I would disagree as evacuation plus a reboot might be expensive.

OP: I once asked if "needrestart" is a good idea, the answer was: "no, ignore it, at least for the PVE services" : https://forum.proxmox.com/threads/proxmox-crashes-reboot-after-upgrading-packages.134670/post-608905

Actually I do restart most of the services - slowly, one by one, with checking the results. My personal understanding is unchanged: after installing updates, the packages are uptodate but the running processes use the old implementation - until they are restarted.

Rebooting is one way to clean up, but not always necessary...

Just my two €¢...

 

[matt.]

Just my two €¢...

Thanks for the helpful comment!

That was what I almost expected, so the confirmation is reassuring.

I would like to add that restarting CEPH services should probably be done with

ceph osd set noout

. I'm doing this for reboots as well, and Proxmox recommends that in various places for node reboots.

 

[sb-jw]

But for a weekly, normal upgrade (executed via "apt dist-upgrade" of course) I would disagree as evacuation plus a reboot might be expensive.

Since I have experienced reboots when setting up the kernel in the past (regardless of whether the node is empty or not), I no longer do anything like that without evacuating the node first. Something can always go wrong, better safe than sorry.

 

[UdoB]

Since I have experienced reboots when setting up the kernel in the past (regardless of whether the node is empty or not), I no longer do anything like that without evacuating the node first. Something can always go wrong, better safe than sorry.

Okay. The own experience is relevant! Once I restarted "watchdog-mux.service" and I had to learn that this would led to a reboot. Since then this one is on my own personal blacklist regarding a restart. (Later that service was configured differently, to avoid this (mis-)behavior officially.)