Using PBS with ceph-client running on PVE with CEPH

fxandrei

Active Member
Jan 10, 2013
132
10
38
Im thinking of using Proxmox Backup Server but im not really sure how to do it so that the backup system is redundant (at least to a certain point)

So right now i have a proxmox cluster with multiple nodes, and uses ceph.

I then have another proxmox cluster that that uses ceph-client to mount the cephfs of the first proxmox cluster.
In the second cluster i have a datastore that is pointing to a directory, that goes to the cephfs on the first cluster (ceph-client mounts the same directory on every node).

So all the backups made on the second cluster are stored on the ceph fs of the first cluster.

Now im thinking of using the PBS.

Would it be ok to have PBS as a vm on the second cluster and use ceph client to mount a directory with a external cephfs ? And then create a datastore on PBS pointing to the cephfs mount point. The PBS vm would move to other nodes in case of problems and the backups would be on a different cluster in case something bad happens.

Would this work ? Im thinking it should.
 

ph0x

Renowned Member
Jul 5, 2020
1,221
193
63
/dev/null
It would probably work but as soon as your Ceph cluster fails all your VM backups are inaccessible ...
 

oversite

Active Member
Jul 13, 2011
148
25
38
Im thinking of using Proxmox Backup Server but im not really sure how to do it so that the backup system is redundant (at least to a certain point)

So right now i have a proxmox cluster with multiple nodes, and uses ceph.

I then have another proxmox cluster that that uses ceph-client to mount the cephfs of the first proxmox cluster.
In the second cluster i have a datastore that is pointing to a directory, that goes to the cephfs on the first cluster (ceph-client mounts the same directory on every node).

So all the backups made on the second cluster are stored on the ceph fs of the first cluster.

Now im thinking of using the PBS.

Would it be ok to have PBS as a vm on the second cluster and use ceph client to mount a directory with a external cephfs ? And then create a datastore on PBS pointing to the cephfs mount point. The PBS vm would move to other nodes in case of problems and the backups would be on a different cluster in case something bad happens.

Would this work ? Im thinking it should.
I am sorry but i don't understand where you are backing up what. For me it sounds really complicated but if you think it should work, give it a go by making a virtual test cluster.
If you want a simple way with redundancy, you can have a physical PBS backup server with physical disks as backup storage. You do backups to this storage from all your proxmox stuff. On your proxmox PVE cluster you make another PBS as VM, give it a storage like ceph or nfs, make it a PBS remote so the standalone PBS server can be synced to the virtual PBS. You now have a redundant PBS AND a redundant backup storage. If your physical PBS server is down, use the PBS VM and the synced storage.
 

fxandrei

Active Member
Jan 10, 2013
132
10
38
I am sorry but i don't understand where you are backing up what. For me it sounds really complicated but if you think it should work, give it a go by making a virtual test cluster.
If you want a simple way with redundancy, you can have a physical PBS backup server with physical disks as backup storage. You do backups to this storage from all your proxmox stuff. On your proxmox PVE cluster you make another PBS as VM, give it a storage like ceph or nfs, make it a PBS remote so the standalone PBS server can be synced to the virtual PBS. You now have a redundant PBS AND a redundant backup storage. If your physical PBS server is down, use the PBS VM and the synced storage.
I trying to find a way to use PBS and CEPH and not have a single point of failure.
 

oversite

Active Member
Jul 13, 2011
148
25
38
I trying to find a way to use PBS and CEPH and not have a single point of failure.
That's good. I just don't follow what you want to backup and where. You have two clusters, one running ceph? or both running ceph? Do you backup VMs from both clusters or just one? Do you plan to have a physical pbs node or only using pbs as a vm?
 

fxandrei

Active Member
Jan 10, 2013
132
10
38
That's good. I just don't follow what you want to backup and where. You have two clusters, one running ceph? or both running ceph? Do you backup VMs from both clusters or just one? Do you plan to have a physical pbs node or only using pbs as a vm?
I know it took a lot to answer this, but i just did not have time.

So yes, both clusters use ceph. Right now the backups from one cluster are made on the cephfs volume that exists on the other cluster.

I would want to use pbs somehow so that i could do incremental backups.
 

oversite

Active Member
Jul 13, 2011
148
25
38
I know it took a lot to answer this, but i just did not have time.

So yes, both clusters use ceph. Right now the backups from one cluster are made on the cephfs volume that exists on the other cluster.

I would want to use pbs somehow so that i could do incremental backups.
I am no expert but if you need booth redundancy/no single point of failure and fast incremental backups, you should run pbs AND pve on the other cluster (or one node of it), with physical disks on the pbs, not the cephfs, and nvme if available. This gives you a good way to optimize the pbs performance and in case of a cluster problem, you can spin up failed VMs on the same box as pbs.
 
Last edited:

fxandrei

Active Member
Jan 10, 2013
132
10
38
I am no expert but if you need booth redundancy/no single point of failure and fast incremental backups, you should run pbs AND pve on the other cluster (or one node of it), with physical disks on the pbs, not the cephfs, and nvme if available. This gives you a good way to optimize the pbs performance and in case of a cluster problem, you can spin up failed VMs on the same box as pbs.
Why use the disks directly on PBS ? That would probably mean i should have a separate server, with raid disks on them, and use them in pbs. That would not be to flexible.
Couldnt i just mount a cephfs in PBS (the one the i use now for backups)?
 

oversite

Active Member
Jul 13, 2011
148
25
38
Why use the disks directly on PBS ? That would probably mean i should have a separate server, with raid disks on them, and use them in pbs. That would not be to flexible.
Couldnt i just mount a cephfs in PBS (the one the i use now for backups)?
It the second ceph cluster is ment for backups only, sure you can do that. How many nodes do you have in the backup cluster? Do you already use PBS in some way?
 
Last edited:

fxandrei

Active Member
Jan 10, 2013
132
10
38
It the second ceph cluster is ment for backups only, sure you can do that. How many nodes do you have in the backup cluster? Do you already use PBS in some way?
The second cluster is used just for storing different stuff like backups and other files, and maybe some vms related to that.

I have not used PBS in any way. Ive just followed and read about it from the moment it was in beta.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!