Distributed storage for backups... Ceph or GlusterFS?

[gkovacs]

We are running three Proxmox nodes, and would like to create a distributed, fault-tolerant filesystem on these nodes for storing the daily backups. First idea was to put a new 3 TB disk in each node, then create a Ceph or GlusterFS volume over them, with 1 disk kept for redundancy and 2 for data (so a single node failure will not affect the backups). Later if/when we install more nodes, we will just add the new disks to the pool.

My questions:
- Ceph or GlusterFS?
- Which one is easier to setup?
- Which one is better performing on three nodes?
- How can I mount / access these from Proxmox? Should I use NFS?
- Can you point me to any tutorials on setting any of these up?

 

[liska_]

In this case you can use just glusterfs, because ceph is stable only as a block storage for now.

Here you have a nice tutorial for two nodes http://www.jamescoyle.net/how-to/435-setup-glusterfs-with-a-replicated-volume-over-2-nodes also check other articles about connecting to proxmox, performance tuning and some others

 

[gkovacs]

In this case you can use just glusterfs, because ceph is stable only as a block storage for now.

You mean you can't mount Ceph over NFS? Why?
Thanks for the link.

 

[vcp_ai]

- How can I mount / access these from Proxmox? Should I use NFS?

If you use glusterfs, you can/must use the Proxmox glusterfs type to mount it:
DataCenter > Storage > Add > glusterfs .
It is already included in Proxmox !!

Also Proxmox has in its repository the last version of glusterfs-server, (it is not in current Debian repositories) so you can be always in the same (last) version as Proxmox glusterfs client.

I'm using same as you propose, but as shared server for VMs (images)

Regards