Why DRBD only supports raw disks

[vkhera]

On the DRBD9 wiki page is this note:

NOTE: DRBD supports only raw disk format at the moment.

What is the reason for this limitation, and does it apply to DRBD8 as well? I did not see such a note on the original DRBD page for current Proxmox VE.

 

[dietmar]

DRBD is a block device (not a file system).

 

[vkhera]

DRBD is a block device (not a file system).

So does that mean you have to use the whole block device as the backing store for a virtual disk? How would you have more than one disk per DRBD if there is no file system?

Clearly I need to find some more material to read up on using DRBD + LVM.

 

[mir]

You create LVM on top of the DRBD block device. And remember that if you use a whole block device it is more convenient to use the raw disk instead of creating partitions since growing the physical volume later will be so much easier.

1) pvcreate /dev/drbd_block_device
2) vgcreate name_of_volume_group /dev/drbd_block_device

Your LVM "filesystem" is ready to provide LVM volumes to VM's.

Eg. You want to create a disk size 10GB to a new VM named server1:
lvcreate -L 10G -n server1_disk1 name_of_volume_group

Read more here: http://www.thegeekstuff.com/2010/08/how-to-create-lvm/

 

[vkhera]

I think I get it now. LVM creates a block device too, but no file system on it. What I need to do is make one lvm per disk image I need in proxmox. If that's the case, how do I restore a VM from a backup? Does it re-create the lvm's for me or do I need to pre-make them?

It seems similar to a ZFS volume created inside a zpool, so why don't we just make a file system on top of the DRBD block device?

 

[mir]

If that is your usecase it is easier to create the LVM from Proxmox GUI. Create the PV and VG from the command line and then go to the storage tab in Proxmox->Datacenter and choose add 'LVM'. In base storage choose your newly create VG. Remember to check 'Shared' to be able to do online migration and snapshot backups.

 

[vkhera]

If that is your usecase it is easier to create the LVM from Proxmox GUI. Create the PV and VG from the command line and then go to the storage tab in Proxmox->Datacenter and choose add 'LVM'. In base storage choose your newly create VG. Remember to check 'Shared' to be able to do online migration and snapshot backups.

If I do it this way, I get the DRBD replicated file system on which I can just create regular qcow2-backed disks? That would be most convenient for me, I think.

Does it impose significant overhead? The use case is for general development servers to test software deploys, office mail server, and our internal ticketing systems. So nothing "production" level.

 

[mir]

If you have a fast disk system base on SAS disks or enterprise SSD's you performance drop will only be measurably.

 

[vkhera]

Thanks! The server is being built with LSI RAID + SAS hotswap drives so it seems this is a good plan for me. This helps me understand quite a lot of how this is put together.

I really appreciate the advice and clarifications provided.

 

[udo]

If I do it this way, I get the DRBD replicated file system on which I can just create regular qcow2-backed disks? That would be most convenient for me, I think.

Does it impose significant overhead? The use case is for general development servers to test software deploys, office mail server, and our internal ticketing systems. So nothing "production" level.

Hi,
if you mount an normal filesystem (xfs,ext4) on both drbdnodes, you will easy destroy your data!!
For such an scenario, you need an cluster filesystem.

Udo

 

[vkhera]

Oh well... I suppose I'll just try to size my LVM's properly so as not to waste too much space and just go with the raw device virtual disks. Or... is there a suitable file system to use here? I'm guessing not.

Do the LVM block devices created on top of DRBD get pre-allocated or are they dynamically allocated like ZFS devices are as needed.

I'm still unclear how one restores a VM from a backup onto raw devices.

 

[vkhera]

I see in http://forum.proxmox.com/threads/21314-pve-zfs-and-drbd that I could build drbd on top of zvols from ZFS running as the native file system. This seems like it would be the best of both worlds where the zvols would not necessarily be fully allocated until the whole thing was written. Much experimenting to do.

 

[mir]

Do the LVM block devices created on top of DRBD get pre-allocated or are they dynamically allocated like ZFS devices are as needed.

LVM does support thin provisioning but this is not implemented in Proxmox. So the answer is they are pre-allocated but if you use ext4 for file system you can start small since ext4 allows for online grow of the file system.

I'm still unclear how one restores a VM from a backup onto raw devices.

Just point to a storage and the restore process will take care of the rest.

 

[mir]

I see in http://forum.proxmox.com/threads/21314-pve-zfs-and-drbd that I could build drbd on top of zvols from ZFS running as the native file system. This seems like it would be the best of both worlds where the zvols would not necessarily be fully allocated until the whole thing was written. Much experimenting to do.

I have no experience with that.