Unable to stop container: operation timed out

bread-baker

Member
Mar 6, 2010
432
0
16
Hello

I have a container running Squeeze + LDAP . It is a test system . I can't stop it:

Code:
vzctl stop 146
Stopping container ...
Unable to stop container: operation timed out
Any suggestions to find out why it does not stop using vzctl?

thanks
Rob
 
yes 21 :

146 21 running 10.100.100.146 ldap.fantinibakery.com


ping does work to it:
proxmox1: ~ # ping -c 3 10.100.100.146
PING 10.100.100.146 (10.100.100.146) 56(84) bytes of data.

--- 10.100.100.146 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2001ms
 
is that log at /var/lib/vz/private/146/var/log ?

logs in /var/lib/vz/private/146/var/log have no clues. logging stipped : Dec 10 02:15 for syslog
 
the file is old:
---------- 1 root root 2734 Nov 22 16:50 /var/lib/vz/private/146/var/log/init.log

tail -n 10 :
Below, you can find the command line options used by this script to
run slapd. Do not forget to specify those options if you
want to look to debugging output:
slapd -g openldap -u openldap -f /etc/ldap/slapd.conf
Starting Name Service Cache Daemon: nscd.
Starting Postfix Mail Transport Agent: postfix.
Starting periodic command scheduler: crond.
Restarting OpenBSD Secure Shell server: sshd.
INIT: no more processes left in this runlevel


are the perms supposed to be '----------' ?
 
I see those perms are normal:

Code:
proxmox1: /var/lib/vz/private/146/var/log # ls -l /var/lib/vz/private/*/var/log/init.log
---------- 1 root root 1327 Nov 19 23:17 /var/lib/vz/private/101/var/log/init.log
---------- 1 root root 1418 Nov 19 23:17 /var/lib/vz/private/102/var/log/init.log
---------- 1 root root 1565 Nov 19 23:17 /var/lib/vz/private/114/var/log/init.log
---------- 1 root root 1566 Nov 19 23:17 /var/lib/vz/private/128/var/log/init.log
---------- 1 root root 1271 Nov 19 23:17 /var/lib/vz/private/129/var/log/init.log
---------- 1 root root 2734 Nov 22 16:50 /var/lib/vz/private/146/var/log/init.log
---------- 1 root root 1440 Dec 12 23:15 /var/lib/vz/private/164/var/log/init.log
---------- 1 root root 1236 Nov 19 23:17 /var/lib/vz/private/169/var/log/init.log
---------- 1 root root 1800 Nov 28 12:42 /var/lib/vz/private/175/var/log/init.log
---------- 1 root root 2670 Dec  9 19:26 /var/lib/vz/private/201/var/log/init.log
---------- 1 root root 1241 Nov 28 11:49 /var/lib/vz/private/2101/var/log/init.log
---------- 1 root root 1285 Nov 28 13:06 /var/lib/vz/private/2159/var/log/init.log
---------- 1 root root 3781 Jun  4  2010 /var/lib/vz/private/307/var/log/init.log
---------- 1 root root 1751 Dec 10 18:31 /var/lib/vz/private/500/var/log/init.log
---------- 1 root root 1654 Dec  8 07:56 /var/lib/vz/private/8001/var/log/init.log
still how do I shut down CT 146 ?

another question - do people generally use open-vz for LDAP ?
 
from ps afx , i searched ldap to find the CT processes

Code:
 4665 ?        Ds     0:07 init [2]
 4731 ?        D      0:07  \_ [init-logger]
 5068 ?        Dl     0:09  \_ /usr/sbin/rsyslogd -c3
 5244 ?        Ds     0:01  \_ /usr/lib/postfix/master
 5247 ?        D      0:00  |   \_ qmgr -l -t fifo -u
24869 ?        R    8979:12  |   \_ pickup -l -t fifo -u -c
29713 ?        D      0:00  |   \_ smtpd -n smtp -t inet -u -c
 5262 ?        Ds     0:01  \_ /usr/sbin/cron
 5274 ?        Ds     0:00  \_ /usr/sbin/sshd -u0
28634 ?        Dsl    0:10  \_ /usr/sbin/slapd -g openldap -u openldap -f /etc/ldap/slapd.conf
14179 ?        D      0:00  \_ /usr/sbin/monit -d 180 -c /etc/monit/monitrc -s /var/lib/monit/monit.state
note the TIME on this line:
24869 ? R 8979:12 | \_ pickup -l -t fifo -u -c


i tried to kill using kill -9 24869 . the process is unkillable.

also top shows very high cpu usage.

I do not want to restart the machine just to stop 146 ..

any suggestions to kill this container?
 
Last edited:
I have the same issue on my production system. While I used the 2.6.24 kernel with PVE 1.6 everything worked fine. Since I have upgraded system to the latest 2.6.32 PVE kernel and ProxMox 1.7, some containers begin to crash in the similiar way. If the problematic container "hangs up", the commands "vzps" and "vztop" also crash with segmentation fault errors. In this cases I am forced to reboot the whole host-machine. Furthermore, I have to reboot it in incorrect way, by turning down its power, because the vz processes refuse to stop gracefully. The command "vzstop" does not have success as well and the process of "vzstop" remains running even the command "kill -9" is issued against it. "vzctl chkpnt $VEID --kill" cant help me too.

I observe this strange behaviour on two different boxes and on random containers. So, this is not hardware troubles. I repeat, before the kernel upgrade from 2.6.24 to 2.6.32 everything was fine. I have met no serious problems with ProxMox/OpenVZ for the two past years. And now, I don't know what to do. Because 2.6.24 is not supported anymore, and 2.6.32 is enough raw and buggy yet.
 
Last edited:
hopefully the cause of the problem is found and fixed.

for the host reboot;
when I use 'init 0' or 'init 6' the system hangs.

I tried 'reboot' and it restarted right away. It could have been something else but reboot worked fast 2 times.
 
Last edited:
I have found this bug in the OpenVZ bug tracker. Here it is: http://bugzilla.openvz.org/show_bug.cgi?id=1626 . Its status is "critical" and developers are already informed. So, we have nothing except be patient to wait when this bug will be fixed. :(

thanks .

By the way, there is another unfixed bug in 2.6.24 kernel, also concerned with NFS: http://bugzilla.openvz.org/show_bug.cgi?id=1295


Are you trying to use NFS in open-vz ? if so , and you probably already know, but there is a lot of info about nfs and open-vz in this and the open-vz forum.

I also remember reading that there are some services which can cause problems in open-vz . now it might be due something i did wrong or a bug.

I do not run these in open-vz , instead use KVM :

nfs
ltsp
ldap
and this reported in the bug you mentioned:
autofs

I also have an EDI program called p2pagent . It needs KVM

Note that for some of those I had the similar issues when using vservers and vmware
 
I always used NFS and LDAP inside my containers and never had any troubles with them, until upgrade to 2.6.32 kernel. I dont like KVM for its disk subsystem penalties and inconvenient ways of maintenance (backup, migration, etc).
 
hello all

I have the same problem sometimes when one of my clients reboot the vm

The only solution:

I recreated the VM and copy the data via ssh :/

Then if the alternative is harder to reboot the nod / server /etc/init.d/vz stop and then start the vm once all down

it is a very great disadvantage, especially in prod

best regards
 
sytry: which kernel are you using?

Code:
Nod6:~# uname -ar

Linux Nod6 2.6.32-4-pve #1 SMP Fri Nov 26 06:42:28 CET 2010 x86_64 GNU/Linux


Nod6:~# pveversion -v
pve-manager: 1.7-10 (pve-manager/1.7/5323)
running kernel: 2.6.32-4-pve
pve-kernel-2.6.32-3-pve: 2.6.32-18
pve-kernel-2.6.32-4-pve: 2.6.32-28
qemu-server: 1.1-25
pve-firmware: 1.0-9
libpve-storage-perl: 1.0-16
vncterm: 0.9-2
vzctl: 3.0.24-1pve4
vzdump: 1.2-9
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1dso1
Nod6:~#

Thks

regards
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!