How to: shared cluster storage for backups

[gkovacs]

At the moment we are running a 2 node PVE 3.1 cluster using local storage for VM's, and a 3rd node hosts an NFS server that is used for backups over gigabit LAN.

We are planning to convert this 3rd node to PVE as well, and would like to create some kind of shared cluster storage for backups with redundancy, using a dedicated disk in every node.
This way if any of the 3 nodes go down, we could still access the backup file system and start restoring to the available nodes.

Is this possible / feasible to do on 3 nodes?

If yes, I would like to ask for some help:
- what kind of cluster file system should we install (GlusterFS, Ceph or something else?)
- should we install it on the PVE host or inside dedicated VM's on every node?
- anything else that comes up regarding that setup from people who have already done something similar

 

[dietmar]

Is this possible / feasible to do on 3 nodes?

Unfortunately, it is not recommended to run a storage server on a compute node (I am not aware of any storage vendor that recommends that).

 

[gkovacs]

Unfortunately, it is not recommended to run a storage server on a compute node (I am not aware of any storage vendor that recommends that).

OK, but is it possible? Since it would only be used for backups and not live VM's, performance is not that important...

GlusterFS has an install procedure for virtual deployments.
http://www.gluster.org/community/documentation/index.php/Getting_started_setup_virt

Let's say I install GlusterFS inside KVM on all threee nodes.
Question is, can I use it from PVE for backups?

 

[dietmar]

Let's say I install GlusterFS inside KVM on all threee nodes.
Question is, can I use it from PVE for backups?

yes, that should work (but it is not recommended).

 

[plofkat]

I have a similar set-up (using moosefs instead of gluster)
5 node cluster, 3 of those nodes are moosefs chunk servers. (I also have independent master and chunk servers that I added later to improve performance)
If you can avoid using the node as storage server you should (you really take a hit on performance when moving large amounts of data)
However, it does work, and very well for that matter.
I have 405 day+ uptime on my nodes, no problems.