Best practice: Upgrade from PVE 8 to 9 with Ceph

[YICT]

Hi, we want to upgrade our cluster from PVE 8 to 9, which also means upgrading Ceph.

Our plan is to move the VMs to local storage first, then upgrade Ceph, and then upgrade the kernel.

Of course, we have image backups, but we have a few questions.

When moving VM storage, will it move the VM to local storage or only the disk? I do not see the option in migration to migrate to local storage, only to another node.

If the VMs are moved to local storage, what is the expected impact and downtime of a kernel update?

Previous updates ran smoothly.

I’d be interested to hear about your experiences and any advice you may have.

 

[Maximiliano]

Hello,

Please take a look at our [pve8to9] and our ceph upgrade guides [1, 2]. Therein you can find our official upgrade guides which contain best practices. Ceph can be upgraded in-place without downtime when following the guides.

[pve8to9] https://pve.proxmox.com/wiki/Upgrade_from_8_to_9
[1] https://pve.proxmox.com/wiki/Ceph_Quincy_to_Reef
[2] https://pve.proxmox.com/wiki/Ceph_Reef_to_Squid

 

[SteveITS]

^that.

How many nodes do you have?

We update PVE/kernels all the time without issue. I’ll post our process shortly.

 

[SteveITS]

Of course, follow the guides for Ceph posted previously. But in general for PVE updates we:

1) in Ceph set checkboxes for:
(any node > Ceph > OSD > Manage Global Flags)
nodeep-scrub
noout
norebalance
norecover
noscrub

2) run ha-manager crm-command node-maintenance enable [I]nodename[/I]
(from https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_maintenance_mode)

3) install updates via web GUI (reboot if needed via web GUI)

4) uncheck "norecover" flag so Ceph can recover (takes a few seconds)

5) run ha-manager crm-command node-maintenance disable nodename to move VMs back again

Once Ceph is 100% green repeat for other nodes (including set norecover, again)

When done unset all the Ceph flags.

 

[YICT]

^that.

How many nodes do you have?

We update PVE/kernels all the time without issue. I’ll post our process shortly.

This environment 3 ,

 

[YICT]

Of course, follow the guides for Ceph posted previously. But in general for PVE updates we:

1) in Ceph set checkboxes for:
(any node > Ceph > OSD > Manage Global Flags)
nodeep-scrub
noout
norebalance
norecover
noscrub

2) run ha-manager crm-command node-maintenance enable [I]nodename[/I]
(from https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_maintenance_mode)

3) install updates via web GUI (reboot if needed via web GUI)

4) uncheck "norecover" flag so Ceph can recover (takes a few seconds)

5) run ha-manager crm-command node-maintenance disable nodename to move VMs back again

Once Ceph is 100% green repeat for other nodes (including set norecover, again)

When done unset all the Ceph flags.

So you do recommend to move the VMS to local storage?

 

[robertlukan]

A very small advice, before rebooting a Ceph node, stop OSDs manually, more info here(why):

https://bugzilla.proxmox.com/show_bug.cgi?id=7827

C

Post in thread 'VM processes stall when rebooting adjacent node'

Jul 19, 2026

Myself as well, when I originally searched this issue I couldn't find much so I assumed it was something on my end but if it seems to be more widespread maybe this thread will gather some attention from the maintainers?

• crlt

 

[SteveITS]

So you do recommend to move the VMS to local storage?

No that will take forever. And require enough storage.

With only 2/3 nodes up Ceph should still function if its storage is not unbalanced. The VMs should stay running. They just need to move off the node being restarted. Which can also be done by using a node shutdown policy now, I just prefer to do it manually.