problem with pct unmount ( rootfs/.lxc_keep, kernel thread)

[joern42]

hi,

i got a problem with unmounting a vm:

# pct list
VMID       Status     Lock         Name
106        running    mounted      panel

# pct unmount 106
umount: /var/lib/lxc/106/rootfs: target is busy.
command 'umount -d /var/lib/lxc/106/rootfs/' failed: exit code 32

so i tried lsof to find out which process is blocking it:

# lsof -w | grep "/var/lib/lxc/106/rootfs"
lxc-start 1066300                               root   12r      REG           147,1018          0      46239 /var/lib/lxc/106/rootfs/.lxc_keep (deleted)

but process 1066300 is not lxc-start, its a kernel thread:

# ps aux | grep 1066300
root     1066300  0.0  0.0   8324  3044 ?        Ss   Jun28   0:21 [lxc monitor] /var/lib/lxc 106

and there is noch lxc-start process running for this vm:

# ps aux | grep lxc-start
#

so what happened? did the lxc-start process died and didnt close its file descriptor?
is there a way to repair that without a restart?

any help is welcome, thanks in advance and regards, joern

edit:

# cat /etc/debian_version
11.7
# cat /proc/version
Linux version 5.15.102-1-pve (build@proxmox) (gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP PVE 5.15.102-1 (2023-03-14T13:48Z)
#

with all updates/upgrades

 

[fiona]

Hi,

# pct list
VMID       Status     Lock         Name
106        running    mounted      panel

did you start the container after mounting it? The container is still running according to this. You need to stop it first, before you can unmount it.

From the man page:

pct mount <vmid>
       Mount the container’s filesystem on the host. This will hold a lock on the container and is meant for emergency maintenance only as it will prevent further operations on the container other than
       start and stop.

it has to be stop, shutdown doesn't work.

I guess the container is privileged, because for unprivileged containers you can unmount it without stopping it first, I think because of namespaces.

 

[joern42]

Hi Fiona,

thanks for your reply. Yes, its a privileged container. I didnt know that pct mount is for emergency only (used before lxc privileged containers and it was mounted all the time).

regards, joern

ps. I found that one: https://github.com/canonical/lxd/issues/11780, maybe it has something to do with this problem, but have to investigate further the next days.

edit: it was mounted while running.