Backup and restore cluster and Ceph

[filippoclikkami]

Hi, i must restore an old 3 node cluster.
Which dirs do i have to backup for the cluster restores exactly like was before (configuration, files...).
Which is the correct procedure to recover it?

Thank you

 

[fabian]

well, if you want to fully back it up, backup all the block devices involved

on the file level it's a bit harder, especially with ceph. you have to considr
- config files
- state files
- systemd things
- actual Ceph data(?)
- installed packages

and it quickly becomes very hard to tell for an outsider that doesn't know your system in-depth what is needed and what isn't..

 

[filippoclikkami]

Maybe exactly like was before is a bit too far. If i want to restore the 3 pve nodes. What is the right way? backup etc dir of every node, reinstall proxmox, restore etc and then recreate cluster? After that, talking about ceph, assuming that i have all disks installed on the nodes like before, do I have to reinstall Ceph and add osds, or there are other dirs where are stored the config files, sys thing..?

 

[fabian]

for PVE itself, if you want to restore it as-is you also need
- /var/lib/pve-cluster (and not /etc/pve, as that is just the mounted version of the DB there - you still might want to keep /etc/pve backed up in case you want to restore individual files from there for some reason)
- any systemd units created by PVE
- any customization you did outside of that

for Ceph you need way more than just the OSDs, they are just the "data stores", you also need monitors and their state, and possibly more, depending on your setup.

the easiest way to achieve what you want is to backup the guests on your old cluster, re-create your config parts that makes sense on the new cluster, and restore the guests there. especially if old and new are different hardware, since then you can re-do your steps if something fails or try a different approach.