Best backup strategy

[proxwolfe]

Hi,

I am about to build a three node PVE cluster (including Ceph) and am wondering what the best backup strategy would be:

Currently, I am running a single node and backup locally to PBS running in a VM on my Synology NAS. This local NAS is, in turn, backing up to a remote NAS.

Once I have the cluster up and running, I will have three local disks (OSDs) that all hold the same content (VMs, LXCs, data, ...) - one per node. This is, essentially, already two local backups of everything. This might eliminate the need for the local PBS instance (and local NAS backup). Instead, it should be sufficient to directly backup to the remote NAS.

On the other hand, I was planning to also backup to a local tape drive (that is supported by PBS). So for this purpose, I still might need the local PBS. If I do keep the PBS locally, I will have three local copies (two via Ceph and one in PBS)? (I would prefer to keep Ceph keeping two local copies (i.e. everything on every node) so that I have a copy in every node, should it need to take over, if the main node dies).

So what would be the best backup strategy for me?

Thanks!

 

[jsterr]

Once I have the cluster up and running, I will have three local disks (OSDs) that all hold the same content (VMs, LXCs, data, ...) - one per node. This is, essentially, already two local backups of everything.

This is not very safe because OSD failure is like a complete host-failure if you only have one osd per node. if you have 2 osds down at the same time your cluster will not work. You should have multiple osds per host.

On the other hand, I was planning to also backup to a local tape drive (that is supported by PBS). So for this purpose, I still might need the local PBS. If I do keep the PBS locally, I will have three local copies (two via Ceph and one in PBS)? (I would prefer to keep Ceph keeping two local copies (i.e. everything on every node) so that I have a copy in every node, should it need to take over, if the main node dies).

Replication != Backup. If you overwrite your data all copies will be affected, if you delete something accidently you wont have backup. Neither RAID nor Replication is Backup.

I would recommend pbs on a separate hw node witih ssds. Dedup will work great if you have similiar vms so you dont need that much space (ssd) because dedup-ratio is high.

 

[proxwolfe]

Replication != Backup. If you overwrite your data all copies will be affected, if you delete something accidently you wont have backup. Neither RAID nor Replication is Backup.

Agree (that's why I referred to the OSDs not as backup but as copies).

And I will have proper backup: Local tape drive backup and remote NAS backup. Against this background, I was hoping to reduce the number of local copies in Ceph and - maybe - eliminate the local NAS backup as well.

This is not very safe because OSD failure is like a complete host-failure if you only have one osd per node. if you have 2 osds down at the same time your cluster will not work. You should have multiple osds per host.

mkay. But it is still safer than only having one stand-alone node, right? I mean, yes, it could be that the main node fails and that an OSD on another node fails at the same time, but this would be materially less likely than the failure of just a node (I'm thinking).

I get your point but I need to strike the right balance between the pursuit of absolute safety on the one hand and complexity and costs on the other (I should have mentioned that this is not an enterprise project for a fortune 500 company but rather "just" my personal data cloud. So while I want my data to be safe and protected against complete loss, a disruption in the availabilty would not be catastrophic).

But let's work this through: How many OSDs should I have per node to be reasonably safe and how many to be "perfectly" safe? How many copies should Ceph keep across my three nodes? Can I tell Ceph to distribute those copies evenly across the nodes (and not keep all on just one node)?

I would recommend pbs on a separate hw node witih ssds.

I had actually contemplated running PBS on one the three nodes (one node will be the main node and the other two (weaker machines), basically, just stand-by - so one could probably assume backup duties while standing by). But you are saying this should be yet another node, right? So I would need at least four HW servers (but at the same time I could eliminate the local NAS).


Thanks