[SOLVED] DRBD9: how to configure multiple volumes?

[firescalp]

Hello,
Fist of all, congratulations to the Proxmox team who did a really great job!

I'm triyng to setup a 2 nodes high avaibility configuration with a 3rd node for quorum with DRBD9, all nodes runs Proxmox 4.1. I found this Wiki page who describes how to accomplish this, and it worked without any troubles.

The point is that I have two types of disks on both nodes:
- 1 RAID5 array with SAS 15k disks to store VM sytem virtual disks
- 1 RAID5 array with SATA disks to store VM data virtual disks

I have to make a DRBD cluster of both RAID arrays, for exemple one called drbd-sas and another called drbd-sata. I triyed many different configuration, but I cant catch the right configuration.
I found this thread, but it didnt help me. I also read the DRBD9 manual but I found nothing interesting for my case.

How can I dot this? Will both volumes be recognized in the WebUI so I can chose the destination volume of the disk when creating a VM or LXC container?

Thanks for your help
Regards

 

[e100]

See: https://forum.proxmox.com/threads/drbd9-multiple-volume-groups.24019/

Also, I've not tested the idea and its not for a novice, but this might work:
Setup DRBD9 according to: https://pve.proxmox.com/wiki/DRBD9 but with two differences:
1. Put both SAS and SATA arrays in the same drbdpool Volume Group
2. Do not use LVM thin provisioning

After creating your VM use pvmove to move the DRBD backing logical volumes onto the specific physical disk in your volume group where you want that disk located.
https://www.centos.org/docs/5/html/Cluster_Logical_Volume_Manager/online_relocation.html

 

[firescalp]

Hi e100, thanks for your help!

I was on the right way. I did create the VG named "drbdpool" with booth SAS and SATA disks. Here is the configuration of all my hosts:

Spoiler: Host 1 Config

Network:
LAN: 192.168.0.102/24
DRBD LAN: 10.0.0.102/24

Pool config:
SAS: /dev/sda4 (300GB wanted)
SATA: /dev/sdb1 (1TB wanted)

Volume Groups:
VG #PV #LV #SN Attr VSize VFree
drbdpool 2 0 0 wz--n- 4.43t 4.43t
pve 1 2 0 wz--n- 111.00g 0

Spoiler: Host 2 Config

Network:
LAN: 192.168.0.101/24
DRBD LAN: 10.0.0.101/24

Pool config:
SAS: /dev/sdc4 (300GB wanted)
SATA: /dev/sda1 (1TB wanted)

Volume Groups:
VG #PV #LV #SN Attr VSize VFree
drbdpool 2 0 0 wz--n- 1.66t 1.66t
pve 1 2 0 wz--n- 103.00g 0

Spoiler: Host 3 Config (For QUORUM)

Network:
LAN: 192.168.0.100/24

You said not to use LVM thin provisioning so I uninstalled the tools "thin-provisioning-tools" and also dependencies drbdmanage. (Is it right?)

Then:
Do I have to create a specific LVM volume per SAS and SATA storage?
Do I also have to create a .res file for DRBD9? If yes, can you help me understand wich options are needed?

 

[firescalp]

Ok, I just figured out how to do this! In fact it was quite easy..

For those who are looking for the same config, here is what you have to do:

• Follow the instructions on the wiki page https://pve.proxmox.com/wiki/DRBD9, but:
  1. Create a VG named "drbdpool" including both SAS and SATA arrays,
  2. Dont create the "drbdthinpool" LV,
  3. After lunching "drbdmanage init" command:
     • Modify config with

       drbdmanage modify-config

     • Choose a storage plugin on this page, and insert it in the config like

       storage-plugin = drbdmanage.storage.lvm.Lvm

       after the [GLOBAL] section,
     • Add other nodes with the "drbdmanage add-node" command
  4. Enjoy your new DRBD config
• When creating a VM, select DRBD storage, install your VM stuff and then, on the primary node run pvmove command to move the disk to the right array, I.E. if I will move the second disk of VM100 to the SATA array (sdb1), in my case it will be:
  
  

  pvmove -n vm-100-disk-2_00 /dev/sda4 /dev/sdb1

  
  You can list all your disks with

  lvs

  .

I hope this will be usefulll to other people!

 

[Kei]

I hope this will be usefulll to other people!

YES!!!!

So let's recap, do we just need to insert that line uncommented and that's it? No reboot?
I've done it and it seems to work amazing, however the strange part is that I didnt need to do the last part (the pvmove). The VM booted just fine anyways. Here's some output, just in case someone can tell me if I'm doing something wrong.

Thank you!!!

 

[firescalp]

Hello Kei

YES!!!!
I've done it and it seems to work amazing, however the strange part is that I didnt need to do the last part (the pvmove)

As I said in my initial post, I have two types of disks on both nodes:
- 1 RAID5 array with SAS 15k disks to store VM sytem virtual disks
- 1 RAID5 array with SATA disks to store VM data virtual disks

So pvemove is needed to move a specific vDisk to a specific disk type (SAS or SATA in my case)

Glad to hear that it was helpful to you!