Setup GlusterFS with Linstor DRBD for High Availability

[andrea_it]

Hello everyone,
I created a 6 nodes cluster running ProxMox 7.4.
High Availability is on.
Every node has 2 nvme ssd dedicated to the storage and 10gbit dedicated connection.
OS runs on different drives.
These ssd are a zraid-1 volume and are part of a DRBD storage pool.

VM disks are stored inside the storage pool, and every disk is replicated on 3 nodes.

Everything works fine and HA is currently guaranteed

Now I want to install Gluster to allow shared access to (some) data across VM.

Gluster volume needs to be HA too so i will create a replicated distributed volume (replica 3)

I though to install Gluster server on 3 new VM. These VM will act exlusively as server.

Then install gluster-fs-client on all the client VMs that needs access to the shared storage.

I excluded to install Gluster on the host nodes even if it's technically possible.

I'm unsure about building a glusterFS volume on top of drbd, it looks like a bit overkill so I just to want to be sure this is the right way of doing this kind of stuff.

I'm a bit concerned about performances due to all these software layers.

The goal is HA and storage sharing across VM without sacrificing performance too much.

EDIT: I already tried Ceph before choosing DRBD, so changing storage is not an option. I'm happy with DRBD.


Thanks in advance for your help

 

[gurubert]

You already have 6 nodes with 2 NVMe each. The recommendation is to use Ceph in such a scenario.

 

[andrea_it]

You already have 6 nodes with 2 NVMe each. The recommendation is to use Ceph in such a scenario.

I already tried Ceph before choosing DRBD. Performances were really poor despite the hardware involved so I switched to drbd.
I'm not asking about alternatives to drbd, I already made my choice and I'm happy with it. Thank you any way for your reply.
I add this reply to my original question.

 

[spirit]

I would advice to not use glusterfs anymore, because it's end of life, and redhat will abandonned it soon
https://access.redhat.com/support/policy/updates/rhs

 

[andrea_it]

I would advice to not use glusterfs anymore, because it's end of life, and redhat will abandonned it soon
https://access.redhat.com/support/policy/updates/rhs

I didn't noticed it before, thanks for the advice.

Any advice/ suggestion other than ceph? Should I take a look at NFSv4?

 

[alexskysilk]

I would advice to not use glusterfs anymore, because it's end of life, and redhat will abandonned it soon
https://access.redhat.com/support/policy/updates/rhs

Not that I am recommending Gluster over ceph, but RHGS # Gluster. FWIW gluster 11 was released in February. I also have no crystal ball to suggest continued active RH involvement in the development- but that doesnt necessarily mean that it will be abandoned/discontinued.

 

[andrea_it]

Not that I am recommending Gluster over ceph, but RHGS # Gluster. FWIW gluster 11 was released in February. I also have no crystal ball to suggest continued active RH involvement in the development- but that doesnt necessarily mean that it will be abandoned/discontinued.

I'm not sure I fully understood relation between Gluster and RedHat.
Gluster is Open Source. RedHat announced EOL at the end of 31/12/2024.
Is Gluster 11 the same version of Red Hat?
So will Gluster die entirely or it wil be mantained by the community?
I don't pretend ypu predict the future, just looking for clarification I'm a bit confused.

Thanks

 

[alexskysilk]

I'm not sure I fully understood relation between Gluster and RedHat.

Red Hat is the nominal owner of gluster. This means that the hold the copyright thereto. They are also the primary user and code contributor.

Gluster is Open Source.

Correct. that means you're free to use, modify, and develop as long as you follow the rules stipulated in its license, which in this case is GPLv3.

RedHat announced EOL at the end of 31/12/2024.

Red Hat announced EOL of Red Hat Gluster Server, a storage product developed and offered by Red Hat.

Is Gluster 11 the same version of Red Hat?

Gluster 11 is the latest iteration of glusterfs. Glusterfs is one of the technologies used in RHGS.

So will Gluster die entirely or it wil be mantained by the community?

It wont die entirely, at least not for quite a while. Maintaining software is hard work, and its common for large scale users to be providing developed code and bug fixes back to the tree. The thing is, Red Hat was the primary user of Gluster; once they cease contributing, other users may decide that its not in their best interest to remain users of the software since it will place larger (too large) of a support burden on them- but not immediately; even if Red Hat end-of-life's RHGS it doesnt mean they stop supporting existing customers; they may be contributing bugfixes for the foreseeable future, and other users (SUSE, etc) may not be able to replace any of their products using the software in the immediate term either- changing underlying software isnt trivial and may take years of development of alternatives.

So how does this affect you?

It really depends on a number of factors, namely:
1. is gluster a technology you already use, understand, and services your use case well?
2. if not, is it preferrable to other solutions for your use case?
3. If yes, how long is the expected lifecycle for your use case? is it short enough to be supported/supportable given what we know about glusters situation?

EDIT: I already tried Ceph before choosing DRBD, so changing storage is not an option. I'm happy with DRBD.

well, there you go.

 

[andrea_it]

@alexskysilk thanks for your detailed answer.
Regarding your questions:

1. is gluster a technology you already use, understand, and services your use case well?

I currently don't use it, I understand it enough, it seems to suits my needs.

2. if not, is it preferrable to other solutions for your use case?

Gluster seems to be the best available know solution. I don't know any other solution apart Ceph (that I discarded) and NFS (that I know really little).

3. How long is the expected lifecycle?

I need to share files across web servers to serve php ecommerce websites, nothing too complex and I don't need bleeding edge tecnology.
Reliability and performance worth more than the last shining object launched in the market.

 

[alexskysilk]

I need to share files across web servers to serve php ecommerce websites, nothing too complex and I don't need bleeding edge tecnology.
Reliability and performance worth more than the last shining object launched in the market.

In that case, you're pursuing the wrong thing. All you need is a simple NAS- a fileserver that is common to all your resources. nfs is most likely the protocol you want, although it you have Windows or Mac guests you can add SMB as well.