Containers fail with "safe_mount_beneath_at: 1106 Function not implemented"

N

neilb

Member
Jul 24, 2017
2
0
6
62
Proxmox 6.3.6

Following a reboot, all VMs re-started without problem but all containers failed.

Initially error was zfs complaining that mount points were missing. zfs list confirmed.

Code: 
root@pve:~# zfs list
NAME                      USED  AVAIL     REFER  MOUNTPOINT
rpool                    3.24T   836G     1.19G  none
rpool/ROOT               3.23T   836G       96K  none
rpool/ROOT/pve-1         3.23T   836G     3.23T  /
rpool/data                 96K   836G       96K  none
rpool/subvol-100-disk-1    96K   836G       96K  none
rpool/subvol-101-disk-1    96K  8.00G       96K  none
rpool/subvol-102-disk-0    96K   836G       96K  none
rpool/subvol-103-disk-1    96K  8.00G       96K  none
rpool/subvol-105-disk-1    96K  30.0G       96K  none
rpool/subvol-106-disk-1    96K  8.00G       96K  none
...

And...
Code: 
root@pve:~# zfs get mountpoint rpool
NAME   PROPERTY    VALUE       SOURCE
rpool  mountpoint  none        local

So...
Code: 
root@pve:~# zfs set mountpoint=/rpool rpool

After which.....
Code: 
root@pve:~# zfs list
NAME                      USED  AVAIL     REFER  MOUNTPOINT
rpool                    3.24T   836G     1.19G  /rpool
rpool/ROOT               3.23T   836G       96K  none
rpool/ROOT/pve-1         3.23T   836G     3.23T  /
rpool/data                 96K   836G       96K  /rpool/data
rpool/subvol-100-disk-1    96K   836G       96K  /rpool/subvol-100-disk-1
rpool/subvol-101-disk-1    96K  8.00G       96K  /rpool/subvol-101-disk-1
rpool/subvol-102-disk-0    96K   836G       96K  /rpool/subvol-102-disk-0
rpool/subvol-103-disk-1   160K  8.00G       96K  /rpool/subvol-103-disk-1
rpool/subvol-105-disk-1    96K  30.0G       96K  /rpool/subvol-105-disk-1
rpool/subvol-106-disk-1   160K  8.00G       96K  /rpool/subvol-106-disk-1
...

All good. But now the containers fail to boot with the error...
Code: 
root@pve:~# pct start 103
__safe_mount_beneath_at: 1106 Function not implemented - Failed to open 51(dev)
__sync_wait: 36 An error occurred in another process (expected sequence number 7)
__lxc_start: 1999 Failed to spawn container "103"
startup for container '103' failed

Similarly, if I clone a container (to ensure a new mountpoint) the new container fails to boot with the same error.

Grateful for thoughts on solving this.

Thanks
 
The debug log suggests that the rootfs is not mounted as /etc cannot be found in the CT.

lxc-start 103 20210421072817.742 INFO lsm - lsm/lsm.c:lsm_init:40 - Initialized LSM security driver AppArmor
lxc-start 103 20210421072817.742 INFO conf - conf.c:run_script_argv:331 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "103", config section "lxc"
lxc-start 103 20210421072818.225 DEBUG conf - conf.c:run_buffer:303 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 103 lxc pre-start produced output: /etc/os-release file not found and autodetection failed, falling back to 'unmanaged'
WARNING: /etc not present in CT, is the rootfs mounted?
got unexpected ostype (unmanaged != ubuntu)
lxc-start 103 20210421072818.234 DEBUG terminal - terminal.c:lxc_terminal_peer_default:672 - Using terminal "/dev/tty" as proxy
lxc-start 103 20210421072818.234 DEBUG terminal - terminal.c:lxc_terminal_winsz:61 - Set window size to 110 columns and 28 rows
lxc-start 103 20210421072818.257 DEBUG seccomp - seccomp.c:parse_config_v2:656 - Host native arch is [3221225534]
lxc-start 103 20210421072818.257 INFO seccomp - seccomp.c:parse_config_v2:807 - Processing "reject_force_umount # comment this to allow umount -f; not recommended"
lxc-start 103 20210421072818.257 INFO seccomp - seccomp.c:do_resolve_add_rule:524 - Set seccomp rule to reject force umounts
lxc-start 103 20210421072818.257 INFO seccomp - seccomp.c:do_resolve_add_rule:524 - Set seccomp rule to reject force umounts
lxc-start 103 20210421072818.257 INFO seccomp - seccomp.c:do_resolve_add_rule:524 - Set seccomp rule to reject force umounts
lxc-start 103 20210421072818.257 INFO seccomp - seccomp.c:parse_config_v2:807 - Processing "[all]"
lxc-start 103 20210421072818.257 INFO seccomp - seccomp.c:parse_config_v2:807 - Processing "kexec_load errno 1"
lxc-start 103 20210421072818.257 INFO seccomp - seccomp.c:do_resolve_add_rule:566 - Adding native rule for syscall[246:kexec_load] action[327681:errno] arch[0]
lxc-start 103 20210421072818.257 INFO seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[246:kexec_load] action[327681:errno] arch[1073741827]
lxc-start 103 20210421072818.257 INFO seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[246:kexec_load] action[327681:errno] arch[1073741886]
lxc-start 103 20210421072818.257 INFO seccomp - seccomp.c:parse_config_v2:807 - Processing "open_by_handle_at errno 1"
lxc-start 103 20210421072818.257 INFO seccomp - seccomp.c:do_resolve_add_rule:566 - Adding native rule for syscall[304:eek:pen_by_handle_at] action[327681:errno] arch[0]
....
(truncated)

Tried mounting the container with pct mount 103. The mountpoint /var/lib/lxc/103/rootfs/ has dev proc and sys directories only...in particular, I believe there should be an empty run directory. However, it is not created.

Assistance still required.

Thanks
 
Last edited:
Hi,

seems that I have a similar problem. I have that error after updating Proxmox and Container. But I don't use ZFS. I can mount the container but it will not start. I posted about that in the german subforum,

Code: 
lxc-start 101 20210423085537.986 ERROR    utils - utils.c:__safe_mount_beneath_at:1106 - Function not implemented - Failed to open 54(dev)

I can mount the container and that is what I see, also the run folder:
Code: 
root@pve:~# pct mount 101
mounted CT 101 in '/var/lib/lxc/101/rootfs'
root@pve:~# ls -l /var/lib/lxc/101/rootfs
total 92
drwxr-xr-x  2 root root  4096 Apr 19 08:45 bin
drwxr-xr-x  2 root root  4096 May 13  2019 boot
drwxr-xr-x  2 root root  4096 Dec 10  2008 dev
drwxr-xr-x 72 root root  4096 Apr 23 15:21 etc
-rw-r--r--  1 root root     0 Apr 23 15:21 fastboot
drwxr-xr-x  2 root root  4096 May 13  2019 home
drwxr-xr-x 10 root root  4096 Jul  8  2019 lib
drwxr-xr-x  2 root root  4096 Jul  8  2019 lib64
drwx------  2 root root 16384 Sep  4  2019 lost+found
drwxr-xr-x  2 root root  4096 Jul  8  2019 media
drwxr-xr-x  3 root root  4096 Sep 14  2019 mnt
drwxr-xr-x  3 root root  4096 Sep 14  2019 opt
drwxr-xr-x  2 root root  4096 May 13  2019 proc
drwx------  8 root root  4096 Apr 19 08:48 root
drwxr-xr-x  3 root root  4096 Jul  8  2019 run
drwxr-xr-x  2 root root  4096 Apr 19 08:45 sbin
drwxr-xr-x  2 root root  4096 Jul  8  2019 srv
drwxr-xr-x  2 root root  4096 May 13  2019 sys
drwxrwxrwt  7 root root  4096 Apr 19 08:45 tmp
drwxr-xr-x 10 root root  4096 Jul  8  2019 usr
drwxr-xr-x 11 root root  4096 Jul  8  2019 var

Code: 
root@pve:~# ls -l /var/lib/lxc/101/rootfs/run/
total 4
drwxrwxrwt 2 root root 4096 Jul  8  2019 lock
-rw-rw-r-- 1 root utmp    0 Jul  8  2019 utmp

BR
Mike
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom