ps aux unable to run

amerikiwi

New Member
Jan 28, 2024
3
0
1
One of the LXC running on ProxMox is unable to shutdown or stop. When I attempt to runi 'ps aux' to see what the PID is to kill the process it is unable to run.

When I run strace ps aux I get the following:

Code:
openat(AT_FDCWD, "/proc/85553/status", O_RDONLY) = 4
read(4, "Name:\tz_rd_int_3\nUmask:\t0000\nSta"..., 2048) = 1045
close(4)                                = 0
openat(AT_FDCWD, "/proc/85553/environ", O_RDONLY) = 4
read(4, "", 131072)                     = 0
close(4)                                = 0
openat(AT_FDCWD, "/proc/85553/cmdline", O_RDONLY) = 4
read(4, "", 131072)                     = 0
close(4)                                = 0
openat(AT_FDCWD, "/proc/85553/ctty", O_RDONLY) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/proc/85554", {st_mode=S_IFDIR|0555, st_size=0, ...}, 0) = 0
openat(AT_FDCWD, "/proc/85554/stat", O_RDONLY) = 4
read(4, "85554 (z_rd_int_2) S 2 0 0 0 -1 "..., 2048) = 173
close(4)                                = 0
openat(AT_FDCWD, "/proc/85554/status", O_RDONLY) = 4
read(4, "Name:\tz_rd_int_2\nUmask:\t0000\nSta"..., 2048) = 1045
close(4)                                = 0
openat(AT_FDCWD, "/proc/85554/environ", O_RDONLY) = 4
read(4, "", 131072)                     = 0
close(4)                                = 0
openat(AT_FDCWD, "/proc/85554/cmdline", O_RDONLY) = 4
read(4, "", 131072)                     = 0
close(4)                                = 0
openat(AT_FDCWD, "/proc/85554/ctty", O_RDONLY) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/proc/85555", {st_mode=S_IFDIR|0555, st_size=0, ...}, 0) = 0
openat(AT_FDCWD, "/proc/85555/stat", O_RDONLY) = 4
read(4, "85555 (z_rd_int_2) S 2 0 0 0 -1 "..., 2048) = 173
close(4)                                = 0
openat(AT_FDCWD, "/proc/85555/status", O_RDONLY) = 4
read(4, "Name:\tz_rd_int_2\nUmask:\t0000\nSta"..., 2048) = 1045
close(4)                                = 0
openat(AT_FDCWD, "/proc/85555/environ", O_RDONLY) = 4
read(4, "", 131072)                     = 0
close(4)                                = 0
openat(AT_FDCWD, "/proc/85555/cmdline", O_RDONLY) = 4
read(4, "", 131072)                     = 0
close(4)                                = 0
openat(AT_FDCWD, "/proc/85555/ctty", O_RDONLY) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/proc/85556", {st_mode=S_IFDIR|0555, st_size=0, ...}, 0) = 0
openat(AT_FDCWD, "/proc/85556/stat", O_RDONLY) = 4
read(4, "85556 (Sonarr) D 85164 85164 851"..., 2048) = 163
close(4)                                = 0
openat(AT_FDCWD, "/proc/85556/status", O_RDONLY) = 4
read(4, "Name:\tSonarr\nUmask:\t0022\nState:\t"..., 2048) = 1501
close(4)                                = 0
openat(AT_FDCWD, "/proc/85556/environ", O_RDONLY

So there is something stopping ps from running. When I run top I get the following:

Code:
top - 18:53:42 up 12:15,  1 user,  load average: 5.43, 5.03, 4.39
Tasks: 886 total,   1 running, 882 sleeping,   2 stopped,   1 zombie
%Cpu(s):  0.8 us,  0.2 sy,  0.0 ni, 95.1 id,  3.1 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :  31988.5 total,  15219.7 free,  15376.6 used,   1930.0 buff/cache     
MiB Swap:      0.0 total,      0.0 free,      0.0 used.  16611.9 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                             
 221732 root      20   0    9.9g   8.0g   5376 S  32.1  25.7   3:45.29 kvm                                                                                                 
   2527 root      20   0  243752 145328   8192 S   1.6   0.4   0:08.58 pvedaemon worke                                                                                     
 224054 root      20   0   12996   4352   2304 R   1.3   0.0   0:00.21 top                                                                                                 
   2534 www-data  20   0  260128 158068   8192 S   0.3   0.5   0:07.35 pveproxy worker                                                                                     
      1 root      20   0  168580   8448   5376 S   0.0   0.0   0:01.90 systemd                                                                                             
      2 root      20   0       0      0      0 S   0.0   0.0   0:00.67 kthreadd                                                                                           
      3 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 rcu_gp                                                                                             
      4 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 rcu_par_gp                                                                                         
      5 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 slub_flushwq                                                                                       
      6 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 netns                                                                                               
      8 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kworker/0:0H-events_highpri                                                                         
     12 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 mm_percpu_wq                                                                                       
     13 root      20   0       0      0      0 I   0.0   0.0   0:00.00 rcu_tasks_kthread                                                                                   
     14 root      20   0       0      0      0 I   0.0   0.0   0:00.00 rcu_tasks_rude_kthread                                                                             
     15 root      20   0       0      0      0 I   0.0   0.0   0:00.00 rcu_tasks_trace_kthread                                                                             
     16 root      20   0       0      0      0 S   0.0   0.0   0:00.40 ksoftirqd/0                                                                                         
     17 root      20   0       0      0      0 I   0.0   0.0   0:14.98 rcu_preempt                                                                                         
     18 root      rt   0       0      0      0 S   0.0   0.0   0:00.15 migration/0                                                                                         
     19 root     -51   0       0      0      0 S   0.0   0.0   0:00.00 idle_inject/0                                                                                       
     20 root      20   0       0      0      0 S   0.0   0.0   0:00.00 cpuhp/0                                                                                             
     21 root      20   0       0      0      0 S   0.0   0.0   0:00.00 cpuhp/1                                                                                             
     22 root     -51   0       0      0      0 S   0.0   0.0   0:00.00 idle_inject/1                                                                                       
     23 root      rt   0       0      0      0 S   0.0   0.0   0:00.40 migration/1                                                                                         
     24 root      20   0       0      0      0 S   0.0   0.0   0:00.34 ksoftirqd/1                                                                                         
     26 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kworker/1:0H-events_highpri                                                                         
     27 root      20   0       0      0      0 S   0.0   0.0   0:00.00 cpuhp/2

So there is a zombie process? Could this be preventing me from stopping the LXC?
 
I'm seeing a lot of RDONLY, are you sure your rootfs didn't remount itself R/O due to errors? Check dmesg and journalctl

You can try ' kill -9 221732 ' but you're probably looking at a hard reboot. If you have SYSRQ enabled, you'll probably have to do something like a REISUB

https://www.kernel.org/doc/html/latest/admin-guide/sysrq.html
If you don't have SysRQ enabled, you can try

echo "1" >/proc/sys/kernel/sysrq