DRBD in Promox 4?

[Jerome Haynes]

Hi guys,

I have no experience with Promox though am planning to move my current infrastructure (Personal dedicated server that runs esxi 6 free) I'm replacing this infrastructure with 3x dedicated servers all with direct attached storage.

One of the main reasons I want to move to promox was DRBD. But looking at the promox wiki page DRBD has been removed. When I was taking a look a few months ago this was one of the main reasons I wanted to switch and I wanted local storage replication rather than having to build a SAN.

I get I could probably achieve this with Ceph or something? However I'd really like to stick with DRBD if possible.

I'm a little confused after looking through some of the threads of whether we can or can't use it or if it is at all supported by promox anymore due to the linbit licensing.

What is the current state of DRBD with promox? Are there any viable alternatives?

Kind Regards,
Jerome Haynes

 

[dietmar]

Most people here use Ceph, because it is more stable and feature complete.

 

[Jerome Haynes]

Hi,

The requirements for Ceph look quite a bit higher looking at the recommended configuration in the wiki. It seems like the 10GB network and SSD's are requirements? I was planning to run DRBD over 1GBE as I'm not running that many VM's and if the performance was suffering I would upgrade to 10GBE as there's no point running 10GBE if I don't need it considering it's quite a bit of a price hike.

So are you saying that I can run Ceph with normal SATA drives? Also it requires 3 identical servers? I was hoping to deploy 2 identical and the third would be a quorum/witness?

Kind Regards,
Jerome Haynes

 

[jeffwadsworth]

@Jerome, you can run CEPH with your 1 GB and regular SATA drives. I would suggest setting up a virtual environment with VirtualBox with 3 Proxmox 4.4 nodes and testing it out. You just won't get great performance using the lesser equipment. It is very IO intensive.

 

[Jerome Haynes]

It's one of the reasons I was more in favour of DRBD as I hear it has the best results even on lesser equipment. If there was a way of running it on two like with DRBD it would be better because I could replace the hardware and I can justify spending the money on upgrading 2x servers but not 3x considering I don't actually need to use the 3rd server for anything. (The amount of VM's I have would be perfect on 2 servers, on 3 it would just be a waste of money.)

I'll definitely give it a test though.

Kind Regards,
Jerome Haynes

 

[Dan Nicolae]

Journal on SSD improve performance of the cluster. Enterprise class SSD is required for better durability. Ceph writes a lot on those journals. With more than 3 servers you will get decent performance. More servers, more drives (OSD) = more IOPS. About the networks, you could bond 2 gigabit interfaces on the Ceph network. It is more cheap than 10gb solutions and for a small cluster is enought. Don't forget to set up corectly pg_num.

 

[e100]

I have had lots of issues with drbd 9 in proxmox 4.x early on, seems to be ok now tho.

In Proxmox 4 the drbd storage plugin used drbdmanage to create individual drbd volumes for each vm disk. I've not followed all the mailing list threads but it looks like linbit plans to maintain the proxmox drbd plugin so drbd is still an option just not an option that proxmox offers/supports.

Also, there are people using drbd in proxmox 4 the same way we have used drbd prior to proxmox 4. Manually setup your drbd volumes using primary/primary mode, slap lvm on top of the drbd volume and add it to proxmox as shared lvm storage.
Last I checked drbd 9 was not safe to use in primary/primary mode so some users have compiled and use drbd 8 instead.

If all you have is 2 or 3 servers, drbd will give you the best performance. But because of the situation created by linbit changing the license on drbdmanage I'm not sure I would recommend using drbd unless it was the only viable option.

 

[Jerome Haynes]

I have had lots of issues with drbd 9 in proxmox 4.x early on, seems to be ok now tho.

In Proxmox 4 the drbd storage plugin used drbdmanage to create individual drbd volumes for each vm disk. I've not followed all the mailing list threads but it looks like linbit plans to maintain the proxmox drbd plugin so drbd is still an option just not an option that proxmox offers/supports.

Also, there are people using drbd in proxmox 4 the same way we have used drbd prior to proxmox 4. Manually setup your drbd volumes using primary/primary mode, slap lvm on top of the drbd volume and add it to proxmox as shared lvm storage.
Last I checked drbd 9 was not safe to use in primary/primary mode so some users have compiled and use drbd 8 instead.

If all you have is 2 or 3 servers, drbd will give you the best performance. But because of the situation created by linbit changing the license on drbdmanage I'm not sure I would recommend using drbd unless it was the only viable option.

Hi e100,

Thank you for your detailed answer. My confusion lies with the new licensing with linbit. Are we allowed to use DRBD9 with Promox 4 anymore? Does it have to come from certain repositories? What permission do we need to have this/how do we gain permission? All the wiki says is about the licensing change and we can't use it but it seems to me people still are?

 

[udo]

Hi e100,

Thank you for your detailed answer. My confusion lies with the new licensing with linbit. Are we allowed to use DRBD9 with Promox 4 anymore? Does it have to come from certain repositories? What permission do we need to have this/how do we gain permission? All the wiki says is about the licensing change and we can't use it but it seems to me people still are?

Hi,
since app. one month linbit provide an repro for drbd:

# cat /etc/apt/sources.list.d/linbit.list
deb http://packages.linbit.com/proxmox/ proxmox-4 drbd-9.0

I use that at home - without trouble yet.

Udo

 

[e100]

That's great and all that linbit providers repo but will that last?
Will they improve the proxmox plugin?
Continue to keep it updated?

Assuming they do all that, when I have an issue and I have some storage related issue on my DRBD backed volume I now have to get support from two companies who could easily point the finger at the other leaving me with no resolution.
Why would I willingly put myself in that position?

They made the license change to prevent others from taking away revenue generated from support subscriptions. This is just speculation but that seems like the sort of move only a failing company would make. Makes me wonder how long the core drbd devs will be around to maintain it.

I do not need 'support' for drbd but I do need access to a repo with stable packages, even willing to pay a reasonable fee for that as I do with Proxmox. When I visit linbit I see they offer support, no pricing is listed, do not see anything equivalent to Proxmox community subscription. If they need revenue they are not making it clear to me how I can pay them a small token in exchange for a stable repo. But they sure have made me think twice about using DRBD.

 

[Jerome Haynes]

That's great and all that linbit providers repo but will that last?
Will they improve the proxmox plugin?
Continue to keep it updated?

Assuming they do all that, when I have an issue and I have some storage related issue on my DRBD backed volume I now have to get support from two companies who could easily point the finger at the other leaving me with no resolution.
Why would I willingly put myself in that position?

They made the license change to prevent others from taking away revenue generated from support subscriptions. This is just speculation but that seems like the sort of move only a failing company would make. Makes me wonder how long the core drbd devs will be around to maintain it.

I do not need 'support' for drbd but I do need access to a repo with stable packages, even willing to pay a reasonable fee for that as I do with Proxmox. When I visit linbit I see they offer support, no pricing is listed, do not see anything equivalent to Proxmox community subscription. If they need revenue they are not making it clear to me how I can pay them a small token in exchange for a stable repo. But they sure have made me think twice about using DRBD.

All of that is my concern. Even if I do use that repo, how long before we are cut off or they say we are breaking license or something.