Hi,
I am testing drbd9 on proxmox in a 3-Node environments and have here an problem when using as underlying device an LVM cached Volume. I followed the instructions for setup drbd9 here https://pve.proxmox.com/wiki/DRBD9, except using a lvm volume cached with 1 SSD on 1 1 HDD. When creating a new VM or restore from a backup I get the following error:
mke2fs 1.42.12 (29-Aug-2014)
The file /dev/drbd/by-res/vm-100-disk-1/0 does not exist and no size was specified.
TASK ERROR: command 'mkfs.ext4 -O mmp -E 'root_owner=0:0' /dev/drbd/by-res/vm-100-disk-1/0' failed: exit code 1
Using the SSD or the HDD separately works without problems.
It seems that there is missing the corresponding block device /dev/drbd100, when using the LVM cached volume.
Following are some drbd and lvm outputs from the failing configuration:
drbdmanage list-assignments
+------------------------------------------------------------------------------------------------------------+
| Node | Resource | Vol ID | | State |
|------------------------------------------------------------------------------------------------------------|
| pve | vm-100-disk-1 | * | | FAILED(3), pending actions: commission |
| pve | vm-100-disk-1 | 0 | | pending actions: commission, attach |
| pve02 | vm-100-disk-1 | * | | FAILED(3), pending actions: commission |
| pve02 | vm-100-disk-1 | 0 | | pending actions: commission, attach |
| pve03 | vm-100-disk-1 | * | | FAILED(3), pending actions: commission |
| pve03 | vm-100-disk-1 | 0 | | pending actions: commission, attach |
+------------------------------------------------------------------------------------------------------------+
drbdmanage list-nodes
+------------------------------------------------------------------------------------------------------------+
| Name | Pool Size | Pool Free | | State |
|------------------------------------------------------------------------------------------------------------|
| pve | 3670016 | 3649531 | | ok |
| pve02 | 3670016 | 3649531 | | ok |
| pve03 | 3670016 | 3649531 | | ok |
+------------------------------------------------------------------------------------------------------------+
drbdsetup status
.drbdctrl role:Secondary
volume:0 disk:UpToDate
volume:1 disk:UpToDate
pve02 role:Secondary
volume:0 peer-disk:UpToDate
volume:1 peer-disk:UpToDate
pve03 role:Secondary
volume:0 peer-disk:UpToDate
volume:1 peer-disk:UpToDate
lvs -a -o +devices
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices
.drbdctrl_0 drbdpool -wi-ao---- 4.00m /dev/sdc1(28240)
.drbdctrl_1 drbdpool -wi-ao---- 4.00m /dev/sdc1(28241)
drbdthinpool drbdpool Cwi-a-C--- 3.50t [dtp_cache] [drbdthinpool_corig] 0.00 13.10 0.00 drbdthinpool_corig(0)
[drbdthinpool_corig] drbdpool owi-aoC--- 3.50t /dev/sdb1(0)
[dtp_cache] drbdpool Cwi---C--- 110.00g 0.00 13.10 0.00 dtp_cache_cdata(0)
[dtp_cache_cdata] drbdpool Cwi-ao---- 110.00g /dev/sdc1(0)
[dtp_cache_cmeta] drbdpool ewi-ao---- 160.00m /dev/sdc1(28160)
[lvol0_pmspare] drbdpool ewi------- 160.00m /dev/sdc1(28200)
data pve -wi-ao---- 158.85g /dev/sda3(16256)
root pve -wi-ao---- 59.50g /dev/sda3(1024)
swap pve -wi-ao---- 4.00g /dev/sda3(0)
I am testing drbd9 on proxmox in a 3-Node environments and have here an problem when using as underlying device an LVM cached Volume. I followed the instructions for setup drbd9 here https://pve.proxmox.com/wiki/DRBD9, except using a lvm volume cached with 1 SSD on 1 1 HDD. When creating a new VM or restore from a backup I get the following error:
mke2fs 1.42.12 (29-Aug-2014)
The file /dev/drbd/by-res/vm-100-disk-1/0 does not exist and no size was specified.
TASK ERROR: command 'mkfs.ext4 -O mmp -E 'root_owner=0:0' /dev/drbd/by-res/vm-100-disk-1/0' failed: exit code 1
Using the SSD or the HDD separately works without problems.
It seems that there is missing the corresponding block device /dev/drbd100, when using the LVM cached volume.
Following are some drbd and lvm outputs from the failing configuration:
drbdmanage list-assignments
+------------------------------------------------------------------------------------------------------------+
| Node | Resource | Vol ID | | State |
|------------------------------------------------------------------------------------------------------------|
| pve | vm-100-disk-1 | * | | FAILED(3), pending actions: commission |
| pve | vm-100-disk-1 | 0 | | pending actions: commission, attach |
| pve02 | vm-100-disk-1 | * | | FAILED(3), pending actions: commission |
| pve02 | vm-100-disk-1 | 0 | | pending actions: commission, attach |
| pve03 | vm-100-disk-1 | * | | FAILED(3), pending actions: commission |
| pve03 | vm-100-disk-1 | 0 | | pending actions: commission, attach |
+------------------------------------------------------------------------------------------------------------+
drbdmanage list-nodes
+------------------------------------------------------------------------------------------------------------+
| Name | Pool Size | Pool Free | | State |
|------------------------------------------------------------------------------------------------------------|
| pve | 3670016 | 3649531 | | ok |
| pve02 | 3670016 | 3649531 | | ok |
| pve03 | 3670016 | 3649531 | | ok |
+------------------------------------------------------------------------------------------------------------+
drbdsetup status
.drbdctrl role:Secondary
volume:0 disk:UpToDate
volume:1 disk:UpToDate
pve02 role:Secondary
volume:0 peer-disk:UpToDate
volume:1 peer-disk:UpToDate
pve03 role:Secondary
volume:0 peer-disk:UpToDate
volume:1 peer-disk:UpToDate
lvs -a -o +devices
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices
.drbdctrl_0 drbdpool -wi-ao---- 4.00m /dev/sdc1(28240)
.drbdctrl_1 drbdpool -wi-ao---- 4.00m /dev/sdc1(28241)
drbdthinpool drbdpool Cwi-a-C--- 3.50t [dtp_cache] [drbdthinpool_corig] 0.00 13.10 0.00 drbdthinpool_corig(0)
[drbdthinpool_corig] drbdpool owi-aoC--- 3.50t /dev/sdb1(0)
[dtp_cache] drbdpool Cwi---C--- 110.00g 0.00 13.10 0.00 dtp_cache_cdata(0)
[dtp_cache_cdata] drbdpool Cwi-ao---- 110.00g /dev/sdc1(0)
[dtp_cache_cmeta] drbdpool ewi-ao---- 160.00m /dev/sdc1(28160)
[lvol0_pmspare] drbdpool ewi------- 160.00m /dev/sdc1(28200)
data pve -wi-ao---- 158.85g /dev/sda3(16256)
root pve -wi-ao---- 59.50g /dev/sda3(1024)
swap pve -wi-ao---- 4.00g /dev/sda3(0)