DRBD9: Multiple volume groups
- Thread starter berner
- Start date
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
I am in the same boat as you, looks like this is not supported directly in Proxmox 4: http://forum.proxmox.com/threads/23968-DRBD9-with-Proxmox-4-and-diverse-storage
WARNING: I've not actually used DRBD9 yet, only read some of the documentation so the following could be complete useless garbage.
This is my understanding of how things are working:
DRBD9 introduces a tool "drbdmanage" that takes care of configuring and replicating config info to the various nodes.
It works by creating DRBD resources on top of an LVM volume Group named "drbdpool" (The pool used is configurable http://drbd.linbit.com/users-guide-9.0/s-dm-set-config.html )
So within Proxmox, when you select to create a VM disk on DRBD it simply calls drbdmanage asking it to create a new DRBD resource for this new VM disk.
Every VM disk is now its own DRBD resource, if created with Proxmox, that is one advantage to this new method.
It appears that drbdmange simply selects the nodes with the most free space when deciding where to store the data.
You can manually use drbdmanage to specify what nodes to use but I am not sure if that is helpful since that would not be integrated with Proxmox.
Ideally Proxmox should provide a method to select the storage nodes too, maybe that will be added some day.
I also suspect you could create the VM disk using Proxmox, then use drbdmanage to migrate the storage to the proper nodes: http://drbd.linbit.com/users-guide-9.0/s-dm-rebalance.html
With DRBD9 you can still define your own DRBD volumes like you did in DRBD8.X.
You just cannot take advantage of the drbdmanage tool to manage such volumes.
It should still be possible to put LVM on top of your custom dual primary DRBD resource and add it as shared LVM storage in Proxmox just like you would have done in 3.X.
WARNING: I've not actually used DRBD9 yet, only read some of the documentation so the following could be complete useless garbage.
This is my understanding of how things are working:
DRBD9 introduces a tool "drbdmanage" that takes care of configuring and replicating config info to the various nodes.
It works by creating DRBD resources on top of an LVM volume Group named "drbdpool" (The pool used is configurable http://drbd.linbit.com/users-guide-9.0/s-dm-set-config.html )
So within Proxmox, when you select to create a VM disk on DRBD it simply calls drbdmanage asking it to create a new DRBD resource for this new VM disk.
Every VM disk is now its own DRBD resource, if created with Proxmox, that is one advantage to this new method.
It appears that drbdmange simply selects the nodes with the most free space when deciding where to store the data.
You can manually use drbdmanage to specify what nodes to use but I am not sure if that is helpful since that would not be integrated with Proxmox.
Ideally Proxmox should provide a method to select the storage nodes too, maybe that will be added some day.
I also suspect you could create the VM disk using Proxmox, then use drbdmanage to migrate the storage to the proper nodes: http://drbd.linbit.com/users-guide-9.0/s-dm-rebalance.html
With DRBD9 you can still define your own DRBD volumes like you did in DRBD8.X.
You just cannot take advantage of the drbdmanage tool to manage such volumes.
It should still be possible to put LVM on top of your custom dual primary DRBD resource and add it as shared LVM storage in Proxmox just like you would have done in 3.X.
Well, looks like dual primary is unsupported until DRBD 9.1: http://drbd.linbit.com/users-guide-9.0/s-enable-dual-primary.html#ftn.idm45543228546144
In the meantime I made some tests. It is indeed possible to use the "old" way of DRBD8 without using drbdmanage at all.
If you use a sensible two-node setup with multiple DRBD volumes you don't even run in permanent dual primary mode under DRBD9. "Sensible setup" means that all VMs on a particular DRBD volume run on the same node. This is recommended for easy recovery from split-brain conditions under DRBD8 as per the wiki.
In that scenario only the active node (running VMs) is primary, the other one is secondary. The new auto-promote feature of DRBD9 automatically promotes the second node to primary when you live migrate VMs to it. As soon as you have migrated all VMs to the second node the first one becomes secondary.
I will test this further but so far everything is working as expected.
If you use a sensible two-node setup with multiple DRBD volumes you don't even run in permanent dual primary mode under DRBD9. "Sensible setup" means that all VMs on a particular DRBD volume run on the same node. This is recommended for easy recovery from split-brain conditions under DRBD8 as per the wiki.
In that scenario only the active node (running VMs) is primary, the other one is secondary. The new auto-promote feature of DRBD9 automatically promotes the second node to primary when you live migrate VMs to it. As soon as you have migrated all VMs to the second node the first one becomes secondary.
I will test this further but so far everything is working as expected.
Do you have to specify just the "auto-promote" option on the resource or is it necessary to also specify "allow-two-primaries" to use the auto-promote feature?
Newbie here on DRBD, but I will need such a setup (2 volumes to easy manage split brain situation, and also have simple vm to be stored in the slower node).
Could you please write a wiki or elaborate further, so I can do some experiment myself (with 2 virtualized proxmox instances)?
Could I achieve the same benefits (low risk of split brain) if I have a 3 node setup, with 3° node being just an Atom PC with no DRBD9 storage and helping for quorum? (a sort of 2 and half node cluster ;P)
Thanks a lot
Could you please write a wiki or elaborate further, so I can do some experiment myself (with 2 virtualized proxmox instances)?
Could I achieve the same benefits (low risk of split brain) if I have a 3 node setup, with 3° node being just an Atom PC with no DRBD9 storage and helping for quorum? (a sort of 2 and half node cluster ;P)
Thanks a lot
Old Way:
https://pve.proxmox.com/wiki/DRBD (NOTE Might require some config file/command syntax changes to work with DRBD9)
Or use new way: https://pve.proxmox.com/wiki/DRBD9
Advantage to new way is its really simple to setup and each virtual disk is its own DRBD resource.
For such a small cluster like you propose I would use this method.
Any small computer that you can install Proxmox VE onto will work for a third quorum node and should work fine even if it does not have DRBD9 data on it.
https://pve.proxmox.com/wiki/DRBD (NOTE Might require some config file/command syntax changes to work with DRBD9)
Or use new way: https://pve.proxmox.com/wiki/DRBD9
Advantage to new way is its really simple to setup and each virtual disk is its own DRBD resource.
For such a small cluster like you propose I would use this method.
Any small computer that you can install Proxmox VE onto will work for a third quorum node and should work fine even if it does not have DRBD9 data on it.