Using PBS with ceph-client running on PVE with CEPH

[fxandrei]

Im thinking of using Proxmox Backup Server but im not really sure how to do it so that the backup system is redundant (at least to a certain point)

So right now i have a proxmox cluster with multiple nodes, and uses ceph.

I then have another proxmox cluster that that uses ceph-client to mount the cephfs of the first proxmox cluster.
In the second cluster i have a datastore that is pointing to a directory, that goes to the cephfs on the first cluster (ceph-client mounts the same directory on every node).

So all the backups made on the second cluster are stored on the ceph fs of the first cluster.

Now im thinking of using the PBS.

Would it be ok to have PBS as a vm on the second cluster and use ceph client to mount a directory with a external cephfs ? And then create a datastore on PBS pointing to the cephfs mount point. The PBS vm would move to other nodes in case of problems and the backups would be on a different cluster in case something bad happens.

Would this work ? Im thinking it should.

 

[ph0x]

It would probably work but as soon as your Ceph cluster fails all your VM backups are inaccessible ...

 

[oversite]

Im thinking of using Proxmox Backup Server but im not really sure how to do it so that the backup system is redundant (at least to a certain point)

So right now i have a proxmox cluster with multiple nodes, and uses ceph.

I then have another proxmox cluster that that uses ceph-client to mount the cephfs of the first proxmox cluster.
In the second cluster i have a datastore that is pointing to a directory, that goes to the cephfs on the first cluster (ceph-client mounts the same directory on every node).

So all the backups made on the second cluster are stored on the ceph fs of the first cluster.

Now im thinking of using the PBS.

Would it be ok to have PBS as a vm on the second cluster and use ceph client to mount a directory with a external cephfs ? And then create a datastore on PBS pointing to the cephfs mount point. The PBS vm would move to other nodes in case of problems and the backups would be on a different cluster in case something bad happens.

Would this work ? Im thinking it should.

I am sorry but i don't understand where you are backing up what. For me it sounds really complicated but if you think it should work, give it a go by making a virtual test cluster.
If you want a simple way with redundancy, you can have a physical PBS backup server with physical disks as backup storage. You do backups to this storage from all your proxmox stuff. On your proxmox PVE cluster you make another PBS as VM, give it a storage like ceph or nfs, make it a PBS remote so the standalone PBS server can be synced to the virtual PBS. You now have a redundant PBS AND a redundant backup storage. If your physical PBS server is down, use the PBS VM and the synced storage.

 

[fxandrei]

I am sorry but i don't understand where you are backing up what. For me it sounds really complicated but if you think it should work, give it a go by making a virtual test cluster.
If you want a simple way with redundancy, you can have a physical PBS backup server with physical disks as backup storage. You do backups to this storage from all your proxmox stuff. On your proxmox PVE cluster you make another PBS as VM, give it a storage like ceph or nfs, make it a PBS remote so the standalone PBS server can be synced to the virtual PBS. You now have a redundant PBS AND a redundant backup storage. If your physical PBS server is down, use the PBS VM and the synced storage.

I trying to find a way to use PBS and CEPH and not have a single point of failure.

 

[oversite]

I trying to find a way to use PBS and CEPH and not have a single point of failure.

That's good. I just don't follow what you want to backup and where. You have two clusters, one running ceph? or both running ceph? Do you backup VMs from both clusters or just one? Do you plan to have a physical pbs node or only using pbs as a vm?

 

[fxandrei]

That's good. I just don't follow what you want to backup and where. You have two clusters, one running ceph? or both running ceph? Do you backup VMs from both clusters or just one? Do you plan to have a physical pbs node or only using pbs as a vm?

I know it took a lot to answer this, but i just did not have time.

So yes, both clusters use ceph. Right now the backups from one cluster are made on the cephfs volume that exists on the other cluster.

I would want to use pbs somehow so that i could do incremental backups.

 

[oversite]

I know it took a lot to answer this, but i just did not have time.

So yes, both clusters use ceph. Right now the backups from one cluster are made on the cephfs volume that exists on the other cluster.

I would want to use pbs somehow so that i could do incremental backups.

I am no expert but if you need booth redundancy/no single point of failure and fast incremental backups, you should run pbs AND pve on the other cluster (or one node of it), with physical disks on the pbs, not the cephfs, and nvme if available. This gives you a good way to optimize the pbs performance and in case of a cluster problem, you can spin up failed VMs on the same box as pbs.

 

[fxandrei]

I am no expert but if you need booth redundancy/no single point of failure and fast incremental backups, you should run pbs AND pve on the other cluster (or one node of it), with physical disks on the pbs, not the cephfs, and nvme if available. This gives you a good way to optimize the pbs performance and in case of a cluster problem, you can spin up failed VMs on the same box as pbs.

Why use the disks directly on PBS ? That would probably mean i should have a separate server, with raid disks on them, and use them in pbs. That would not be to flexible.
Couldnt i just mount a cephfs in PBS (the one the i use now for backups)?

 

[oversite]

Why use the disks directly on PBS ? That would probably mean i should have a separate server, with raid disks on them, and use them in pbs. That would not be to flexible.
Couldnt i just mount a cephfs in PBS (the one the i use now for backups)?

It the second ceph cluster is ment for backups only, sure you can do that. How many nodes do you have in the backup cluster? Do you already use PBS in some way?

 

[fxandrei]

It the second ceph cluster is ment for backups only, sure you can do that. How many nodes do you have in the backup cluster? Do you already use PBS in some way?

The second cluster is used just for storing different stuff like backups and other files, and maybe some vms related to that.

I have not used PBS in any way. Ive just followed and read about it from the moment it was in beta.

 

[fxandrei]

So just an update.
This was easier than i was thinking.

So i installed a debian 11 container on the backup cluster.
I installed pbs and added a mount point inside it that was pointing to a folder in the cephfs instance i had it the backup cluster.
I then created a datastore in a directory in the mount point.

I then simply connected to the pbs instance from the main cluster .

When i make a backup its being saved in the a pbs datastore that is in a directory the cephfs on the backup cluster.

But there is one thing that i have not tried.
Lets say the pbs instance goes out the window somehow. If i install a new instance, could i use that datastore directory ?

Would there be any problems related to the fact that the datastore was created by some other pbs instance ?

 

[ph0x]

No, they're interchangeable.

 

[fxandrei]

No, they're interchangeable.

Thats great.
Thanks.