VM can't start on DRBD9: Could not open '/dev/drbd/by-res/vm-x-disk-1/0': Permission denied

[brickmasterj]

After a system crash I needed to restart a VM, call it ID x, after one of the other nodes in the DRBD9 cluster derped out. Now, node 1 derped out, and the VM was running on node 2, the issue being that restarting the VM on either node returns a KVM-error: Could not open '/dev/drbd/by-res/vm-x-disk-1/0': Permission denied. The output of 'drbdadm status' is also remarkable, for some reason vm-x-disk-1 is outdated on node 1 and 2 but 3 is set to 'StandAlone'. So why does it say that the data is outdated even though the VM was running on a node that was never offline? Does this mean the data got corrupted by the other node derping out or is it save to say that the data on node 2 is still the up to date data, and if so, how do I let DRBD know that it is (cause I'm assuming that would fix the issue)?

 

[finish06]

Were you ever able to correct this problem? I am experiencing a very similar issue. I had a power failure which resulted with two of three machines in cluster powering down. The third machine is for quorum only and doesn't contain a DRBD disk. Now, I am unable to start any of my VMs which were on the DRBD cluster. I get a permission denied error.

Any and all help grateful!

Thanks!

 

[brickmasterj]

Were you ever able to correct this problem? I am experiencing a very similar issue. I had a power failure which resulted with two of three machines in cluster powering down. The third machine is for quorum only and doesn't contain a DRBD disk. Now, I am unable to start any of my VMs which were on the DRBD cluster. I get a permission denied error.

Any and all help grateful!

Thanks!

Unfortunately no. We ended up ditching DRBD9 altogether in favour of a combination of snapshots and other live migration tools. To this day I would like to know what was causing this but at this rate, we might just wait for a bit and see if it gets fixed or it becomes a wider known issue. For now I'm sorry that I'm unable to help you because it really sucked.

 

[finish06]

Unfortunately no. We ended up ditching DRBD9 altogether in favour of a combination of snapshots and other live migration tools. To this day I would like to know what was causing this but at this rate, we might just wait for a bit and see if it gets fixed or it becomes a wider known issue. For now I'm sorry that I'm unable to help you because it really sucked.

I was finally able to get my VMs back by setting the primary on each VM in /dev/drbdpool/. However, it has happened again and I was able to successfully get my VMs running except for one, so I just used a backup of said VM. However, when I tried to remove the hosed VM, I am unable to delete it from the DRBD pool. I deleted the VM from the GUI however the image is still present. So confusing... anyone have any advice on where to move from here?

 

[Kei]

So confusing... anyone have any advice on where to move from here?

Same problem. Anyone can help?