ceph upgrade

[walter.egosson]

Hi!
I have proxmox 4 on three nodes and ceph hammer on each:
I want to upgrade ceph from hammer to jewel and then from jewel to hammer. Since the upgrade is done node by node, will there be a risk during the process while some nodes will run ceph hammer and the others ceph jewel (those being upgraded)?
thx

 

[dcsapak]

jewel to hammer.

i hope you mean luminous?

also did you see these sites:
https://pve.proxmox.com/wiki/Ceph_Hammer_to_Jewel

https://pve.proxmox.com/wiki/Ceph_Jewel_to_Luminous

note that luminous is only supported on pve 5.x (but you must upgrade on 4.x before upgrading to 5, this is also written on the upgrade notes)

 

[walter.egosson]

Thank you a lot,
The upgrade notes seems incomplete to me.

Yes, I meant jewel to luminous. Talking about ceph hammer to jewel step: I have some worries if you could help me :
- When upgrading Ceph since it is done node by node , do I have to shutdown the VMs/CTs on the to-be-upgraded proxmox node then upgrade that node or can I migrate them on the two other nodes, let them run and finally migrate them back to the upgraded proxmox node?
- since the upgrade process is progressive, there will be a moment when I will have 2 ceph jewel and 1 ceph hammer coexisting => will my data be lost?

Thanks!

i hope you mean luminous?

also did you see these sites:
https://pve.proxmox.com/wiki/Ceph_Hammer_to_Jewel

https://pve.proxmox.com/wiki/Ceph_Jewel_to_Luminous

note that luminous is only supported on pve 5.x (but you must upgrade on 4.x before upgrading to 5, this is also written on the upgrade notes)

.

i hope you mean luminous? also did you see these sites: https://pve.proxmox.com/wiki/Ceph_Hammer_to_Jewel https://pve.proxmox.com/wiki/Ceph_Jewel_to_Luminous note that luminous is only supported on pve 5.x (but you must upgrade on 4.x before upgrading to 5, this is also written on the upgrade notes)

 

[dcsapak]

just follow the upgrade guide node by node (this is also mentioned there)

if you are unsure, maybe setup a non-production system where you can test the upgrade

as a general rule: always have backups (and verify they work)

 

[walter.egosson]

We ended up using a 6TB NFS storage and move all VM storage there to leave the ceph unused and NO DOWNTIME
Everything went straight through, absolutely no issue
If it can help others:
Our NFS storage had 1Gb interface (125MB/s maximum theorical speed) was able to handle almost 50 VMs hard-drives (with 25MB/sec write and 10MB/sec read average) leaving a confortable bandwitdth margin (absolute max was ~110MB/sec)

1/ connect NFS storage (or any non-ceph storage you have)
2/ live migrate hard-drives to that storage (no downtime but slightly longer that offline migrate)
3/ upgrade your cluster so that is your ceph cluster messes up, your VMs will not be affected