VM can't start on DRBD9: Could not open '/dev/drbd/by-res/vm-x-disk-1/0': Permission denied

brickmasterj

New Member
Oct 18, 2015
19
1
3
After a system crash I needed to restart a VM, call it ID x, after one of the other nodes in the DRBD9 cluster derped out. Now, node 1 derped out, and the VM was running on node 2, the issue being that restarting the VM on either node returns a KVM-error: Could not open '/dev/drbd/by-res/vm-x-disk-1/0': Permission denied. The output of 'drbdadm status' is also remarkable, for some reason vm-x-disk-1 is outdated on node 1 and 2 but 3 is set to 'StandAlone'. So why does it say that the data is outdated even though the VM was running on a node that was never offline? Does this mean the data got corrupted by the other node derping out or is it save to say that the data on node 2 is still the up to date data, and if so, how do I let DRBD know that it is (cause I'm assuming that would fix the issue)?
 
Were you ever able to correct this problem? I am experiencing a very similar issue. I had a power failure which resulted with two of three machines in cluster powering down. The third machine is for quorum only and doesn't contain a DRBD disk. Now, I am unable to start any of my VMs which were on the DRBD cluster. I get a permission denied error.

Any and all help grateful!

Thanks!
 
Were you ever able to correct this problem? I am experiencing a very similar issue. I had a power failure which resulted with two of three machines in cluster powering down. The third machine is for quorum only and doesn't contain a DRBD disk. Now, I am unable to start any of my VMs which were on the DRBD cluster. I get a permission denied error.

Any and all help grateful!

Thanks!
Unfortunately no. We ended up ditching DRBD9 altogether in favour of a combination of snapshots and other live migration tools. To this day I would like to know what was causing this but at this rate, we might just wait for a bit and see if it gets fixed or it becomes a wider known issue. For now I'm sorry that I'm unable to help you because it really sucked.
 
Unfortunately no. We ended up ditching DRBD9 altogether in favour of a combination of snapshots and other live migration tools. To this day I would like to know what was causing this but at this rate, we might just wait for a bit and see if it gets fixed or it becomes a wider known issue. For now I'm sorry that I'm unable to help you because it really sucked.

I was finally able to get my VMs back by setting the primary on each VM in /dev/drbdpool/. However, it has happened again and I was able to successfully get my VMs running except for one, so I just used a backup of said VM. However, when I tried to remove the hosed VM, I am unable to delete it from the DRBD pool. I deleted the VM from the GUI however the image is still present. So confusing... anyone have any advice on where to move from here?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!