>2 DRBD node setup

[svendsen]

Hi,

Does any have experience setting up >2 DRBD nodes?

I want 3 node primary-primary drbd cluster + replication to 4th node in primary-secondary mode.

Is that possible to do with DRBD? Or should I investigate possibilities with fx Gluster or Sheepdog?


- Michael

 

[e100]

DRBD can replicate to two nodes for primary/primary.

You can stack DRBD devices http://www.drbd.org/users-guide/s-three-nodes.html
With that I think you would end up with something like primary/primary/secondary

You could also have multiple DRBD volumes, each one on only two nodes in primary/primary.
Nodes A, B and C:
DRBD1 is primary/primary on A and B
DRBD2 is primary/primary on B and C
DRBD3 is primary/primary on C and A

In the Proxmox GUI when adding the shared storage you can select the specific nodes where a particular DRBD volume is accessible.

I tested sheepdog with Proxmox 2.0 Beta, but not with the 2.0 release.
It worked but was much slower than DRBD, used lots of CPU and was difficult to add disks to VMs(no Sheepdog support in Proxmox GUI yet)

I also tested Gluster with 2.0 Beta.
It also worked but again was slower than DRBD.

Also no snapshot backup with sheepdog or gluster yet.

 

[svendsen]

Thanks for your very nice reply!

Do you have experience with stacking DRBD?

Also, I read somewhere that protocol C is mandatory in primary-primary mode. Is this correct? Or could you use B or even A in certain circumstances?

Thanks for your input on Sheepdog+Gluster - that's also what I've read elsewhere..

 

[e100]

Never stacked DRBD and Protocol C is necessary for primary/primary.

 

[svendsen]

Ok, thanks for your help so far

ONE last question. Would it be better idea to, in a 3-4-node setup, to make one big DRBD storage between node A and B, and setup a NFS server using whole storage, and configure all other VMs to use that NFS server as storage?
In that way you can have 3, 4, 5,... Proxmox nodes with access to same data, and thus having live migration and everything from all nodes.

What is your opinion?

 

[svendsen]

Bump

 

[e100]

There are how to articles all over the net on how to do HA NFS with DRBD. So yes you can do that if you want.

The only benefit is that you can have multiple Proxmox servers using the HA NFS as you mentioned.

You end up with a more complex setup, slower disk IO and loose the ability to do snapshot backups.

You give up quite a bit for little benefit.