VM is locked , Cannot unlock

danitfk · Jul 10, 2015

Hello all ,

I've faced to problem with starting one my VM's in proxmox , before that I did turn off dedicated server (of curse before shutdown I turned off all CT's and VM's) then I returned back server and When I tried to start VM I got this error: VM is locked (backup) so I did try qm unlock VMID but I got an error with following this message:

Code:

unable to open file '/etc/pve/nodes/node1/qemu-server/102.conf.tmp.252157' - Permission denied

At first , When I run the hypervisor's node I did setup a cluster with Two-node but after that I did turned off second node and run this command for solve quorum error :

Code:

pvecm expected 1

Then server works just fine , But after that reboot I cannot start one of my VM's because of this error VM is locked (backup) and Now I cannot backup or unlock any of my VM's becuase of this error:

Code:

[B]unable to open file '/etc/pve/nodes/node1/qemu-server/102.conf.tmp.252157' - Permission denied
[/B]

Any suggestion please?

udo · Jul 10, 2015

danitfk said:
Hello all ,

I've faced to problem with starting one my VM's in proxmox , before that I did turn off dedicated server (of curse before shutdown I turned off all CT's and VM's) then I returned back server and When I tried to start VM I got this error: VM is locked (backup) so I did try qm unlock VMID but I got an error with following this message:

Code:

Code:

unable to open file '/etc/pve/nodes/node1/qemu-server/102.conf.tmp.252157' - Permission denied

At first , When I run the hypervisor's node I did setup a cluster with Two-node but after that I did turned off second node and run this command for solve quorum error :

Code:

pvecm expected 1

Then server works just fine , But after that reboot I cannot start one of my VM's because of this error VM is locked (backup) and Now I cannot backup or unlock any of my VM's becuase of this error:

Code:

Code:

[B]unable to open file '/etc/pve/nodes/node1/qemu-server/102.conf.tmp.252157' - Permission denied [/B]

Any suggestion please?

Hi,
are both nodes running?
Do you have quorum? is /etc/pve is writable?

Code:

touch /etc/pve/dummy
rm /etc/pve/dummy

How looks your quorum?

Code:

pvecm status

Udo

danitfk · Jul 11, 2015

udo said:
Hi,
are both nodes running?
Do you have quorum? is /etc/pve is writable?

Code:

touch /etc/pve/dummy rm /etc/pve/dummy

How looks your quorum?

Code:

pvecm status

Udo

Hi Udo ,

No just one of nodes is running , like before.
Permission denied When I tried to make dummy file, also pvecm is stopped because cman is not running and cman cannot start because of this error:

Code:

Starting cluster: 
   Checking if cluster has been disabled at boot... [  OK  ]
   Checking Network Manager... [  OK  ]
   Global setup... [  OK  ]
   Loading kernel modules... [  OK  ]
   Mounting configfs... [  OK  ]
   Starting cman... Cannot find node name in cluster.conf
Unable to get the configuration
Cannot find node name in cluster.conf
cman_tool: corosync daemon didn't start Check cluster logs for details
[FAILED]

danitfk · Jul 11, 2015

Also as I mentioned before , I decluster servers a long time ago

and here is my cluster.conf now:

Code:

<?xml version="1.0"?>
<cluster name="[B]Dummy Name[/B]" config_version="4">


  <cman keyfile="/var/lib/pve-cluster/corosync.authkey">
  </cman>


  <clusternodes>


  </clusternodes>


</cluster>

It's seems so empty, right? ^_^

Also It's not possible to make changes in ssh folder , same error for writing permission and cannot chand chmod of files.

Also It's my mount details:

Code:

root@node1:/etc/ssh# df -h
Filesystem               Size  Used Avail Use% Mounted on
udev                      10M     0   10M   0% /dev
tmpfs                    6.7G  428K  6.7G   1% /run
/dev/mapper/pve-root      95G   51G   40G  57% /
tmpfs                    5.0M     0  5.0M   0% /run/lock
tmpfs                     14G   31M   14G   1% /run/shm
/dev/fuse                 30M   24K   30M   1% /etc/pve
/dev/mapper/pve-data     1.7T  300G  1.4T  19% /var/lib/vz
/dev/sdb1                1.8T  1.6T  216G  89% /sdb
/var/lib/vz/private/109   50G  1.8G   49G   4% /var/lib/vz/root/109
none                     3.0G  4.0K  3.0G   1% /var/lib/vz/root/109/dev
none                     4.0K     0  4.0K   0% /var/lib/vz/root/109/sys/fs/cgroup
none                     615M  112K  615M   1% /var/lib/vz/root/109/run
none                     5.0M     0  5.0M   0% /var/lib/vz/root/109/run/lock
none                     3.0G     0  3.0G   0% /var/lib/vz/root/109/run/shm
none                     100M     0  100M   0% /var/lib/vz/root/109/run/user

udo · Jul 11, 2015

danitfk said:
Also as I mentioned before , I decluster servers a long time ago
and here is my cluster.conf now:

Code:

<?xml version="1.0"?> <cluster name="[B]Dummy Name[/B]" config_version="4"> <cman keyfile="/var/lib/pve-cluster/corosync.authkey"> </cman> <clusternodes> </clusternodes> </cluster>

It's seems so empty, right? ^_^
...

Hi,
your "declustering" was wrong! An pve-non-cluster-system don't have an /etc/pve/cluster.conf.

The official way is an new installation... but I have seen a long time ago an thread about successfull declustering.

Udo

danitfk · Jul 11, 2015

udo said:
Hi,
your "declustering" was wrong! An pve-non-cluster-system don't have an /etc/pve/cluster.conf.

The official way is an new installation... but I have seen a long time ago an thread about successfull declustering.

Udo

So clustering in PVE is a one way road and cannot return ?

tom · Jul 11, 2015

danitfk said:
So clustering in PVE is a one way road and cannot return ?

if you have a single node now, just delete the cluster.conf.

> rm /etc/pve/cluster.conf
> rm -r /etc/cluster
> reboot

danitfk · Jul 11, 2015

tom said:
if you have a single node now, just delete the cluster.conf.

> rm /etc/pve/cluster.conf
> rm -r /etc/cluster
> reboot

Dear Tom ,
Thanks for your reply but the problem is , I cannot create / remove / modify any files in /etc.

tom · Jul 11, 2015

you cannot do the following?:

> rm -r /etc/cluster

danitfk · Jul 11, 2015

tom said:
you cannot do the following?:

> rm -r /etc/cluster

Here for cluster.conf

Code:

root@node1:~# rm -r /etc/pve/cluster.conf
rm: cannot remove `/etc/pve/cluster.conf': Permission denied

but rm /etc/cluster is ok and now I checked there is no /etc/cluster anymore.

Also my biggest problem is I cannot unlock the VM , I'm getting this error:

Code:

root@node1:~# qm unlock 103
unable to open file '/etc/pve/nodes/node1/qemu-server/103.conf.tmp.416266' - Permission denied

tom · Jul 11, 2015

try to stop cman:

> service stop cman

remove the cluster.conf from /etc/pve:

> rm /etc/pve/cluster.conf

reboot.

danitfk · Jul 11, 2015

tom said:
try to stop cman:

> service stop cman

remove the cluster.conf from /etc/pve:

> rm /etc/pve/cluster.conf

reboot.

Hello ,

When I run this command:

Code:

service cman stop

There is no output , and about remove cluster.conf , I got this error:

Code:

root@node1:/etc/pve# rm -rf cluster.conf
rm: cannot remove `cluster.conf': Permission denied

As I mentioned before , I cannot append / add / remove any thing in /etc/.

tom · Jul 11, 2015

danitfk said:
Hello ,

When I run this command:

Code:

service cman stop

There is no output , and about remove cluster.conf , I got this error:

Code:

root@node1:/etc/pve# rm -rf cluster.conf rm: cannot remove `cluster.conf': Permission denied

As I mentioned before , I cannot append / add / remove any thing in /etc/.

you CAN edit everything in /etc/, but not inside /etc/pve/...
read more on: http://pve.proxmox.com/wiki/Proxmox_Cluster_file_system_(pmxcfs)

as its hard to fix broken cluster via forum support and its probably faster for you to just re-install.

or you go for a support package (with remote login) so our support team can fix it for you.

danitfk · Jul 11, 2015

tom said:
you CAN edit everything in /etc/, but not inside /etc/pve/...
read more on: http://pve.proxmox.com/wiki/Proxmox_Cluster_file_system_(pmxcfs)

as its hard to fix broken cluster via forum support and its probably faster for you to just re-install.

or you go for a support package (with remote login) so our support team can fix it for you.

Yes , I'm sorry.
But I tried pmxcfs too.

Code:

root@node1:/etc/ssh# [B]pmxcfs -l[/B]
[main] notice: unable to aquire pmxcfs lock - trying again
[main] crit: unable to aquire pmxcfs lock: Resource temporarily unavailable
[main] notice: exit proxmox configuration filesystem (-1)

absolutely I'll get a subscription package but I have some problem(s) in here.

udo · Jul 11, 2015

danitfk said:
Yes , I'm sorry.
But I tried pmxcfs too.

Code:

root@node1:/etc/ssh# [B]pmxcfs -l[/B] [main] notice: unable to aquire pmxcfs lock - trying again [main] crit: unable to aquire pmxcfs lock: Resource temporarily unavailable [main] notice: exit proxmox configuration filesystem (-1)

absolutely I'll get a subscription package but I have some problem(s) in here.

Hi,
that sounds, that you have /etc/pve still mounted? Try to umount first...

Udo

danitfk · Jul 12, 2015

udo said:
Hi,
that sounds, that you have /etc/pve still mounted? Try to umount first...

Udo

Yes , It's mounted.

Code:

/dev/fuse                 30M   24K   30M   1% /etc/pve

I must umount /dev/fuse from server ?

BTW I'm very very appreciate for your replies.

danitfk · Jul 12, 2015

Fixed:
I fixed the cman then unlock the VM.
Love you guys.

udo · Jul 12, 2015

danitfk said:
Fixed:
I fixed the cman then unlock the VM.
Love you guys.

Hi,
fine that all work now!

Udo

Search

Search

VM is locked , Cannot unlock

danitfk

Member

udo

Distinguished Member

danitfk

Member

danitfk

Member

udo

Distinguished Member

danitfk

Member

tom

Proxmox Staff Member

danitfk

Member

tom

Proxmox Staff Member

danitfk

Member

tom

Proxmox Staff Member

danitfk

Member

tom

Proxmox Staff Member

danitfk

Member

udo

Distinguished Member

danitfk

Member

danitfk

Member

udo

Distinguished Member

We value your privacy