[SOLVED] Some services fail to start, trying to set up mount namespacing

[grobs]

Hi everyone,

I'm currently struggling with a blocking issue.

I'm running proxmox 6 (proxmox-ve: 6.1-2 / kernel 5.3.10-1-pve) and created a container running Debian Buster from the latest standard template on pveam (debian-10.0-standard_10.0-1_amd64.tar.gz).

The problem is that some services won't start at all (and even install) due to a namespacing issue.

Example 1 with munin (refusing to install):

# apt install munin-node
[...]
Restarting munin-node..Job for munin-node.service failed because the control process exited with error code.
See "systemctl status munin-node.service" and "journalctl -xe" for details.
invoke-rc.d: initscript munin-node, action "start" failed.
● munin-node.service - Munin Node
   Loaded: loaded (/lib/systemd/system/munin-node.service; disabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Thu 2020-02-06 14:08:23 UTC; 5ms ago
     Docs: man:munin-node(1)
           http://munin.readthedocs.org/en/stable-2.0/reference/munin-node.html
  Process: 3777 ExecStart=/usr/sbin/munin-node $DAEMON_ARGS (code=exited, status=226/NAMESPACE) ==========> Here is the important info
dpkg: erreur de traitement du paquet munin-node (--configure) :
installed munin-node package post-installation script subprocess returned error exit status 1
Traitement des actions différées (« triggers ») pour man-db (2.8.5-2) ...
Traitement des actions différées (« triggers ») pour systemd (241-7~deb10u2) ...
Des erreurs ont été rencontrées pendant l'exécution :
munin-node
[...]
E: Sub-process /usr/bin/dpkg returned an error code (1)

# dpkg -l | grep munin-node
iF  munin-node                     2.0.49-1                      all          network-wide graphing framework (node)

The package is half-installed as the postscript failed, trying to start the service.

Example 2 with mysql / mariadb (refusing to install):

# apt install default-mysql-server
[...]
Job for mariadb.service failed because the control process exited with error code.
See "systemctl status mariadb.service" and "journalctl -xe" for details.
[...]

# journalctl -xe
[...]
févr. 06 14:21:19 vs-mail systemd[6376]: mariadb.service: Failed to set up mount namespacing: Permission denied
févr. 06 14:21:19 vs-mail systemd[6376]: mariadb.service: Failed at step NAMESPACE spawning /usr/bin/install: Permission denied
[...]
-- Le processus /usr/bin/install n'a pas pu être exécuté, et a donc échoué.
-- Le code d'erreur renvoyé est ERRNO.
févr. 06 14:21:19 vs-mail systemd[1]: mariadb.service: Control process exited, code=exited, status=226/NAMESPACE
[...]
-- An ExecStartPre= process belonging to unit mariadb.service has exited.
-- The process' exit code is 'exited' and its exit status is 226.
févr. 06 14:21:19 vs-mail systemd[1]: mariadb.service: Failed with result 'exit-code'.
[...]
-- The unit mariadb.service has entered the 'failed' state with result 'exit-code'.
févr. 06 14:21:19 vs-mail systemd[1]: Failed to start MariaDB 10.3.18 database server.
-- Subject: L'unité (unit) mariadb.service a échoué

# service mysql status
● mariadb.service - MariaDB 10.3.18 database server
   Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 2020-02-06 14:28:24 UTC; 2s ago
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/
  Process: 6731 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=226/NAMESPACE)

févr. 06 14:28:24 vs-mail systemd[1]: Starting MariaDB 10.3.18 database server...
févr. 06 14:28:24 vs-mail systemd[6731]: mariadb.service: Failed to set up mount namespacing: Permission denied
févr. 06 14:28:24 vs-mail systemd[6731]: mariadb.service: Failed at step NAMESPACE spawning /usr/bin/install: Permission denied
févr. 06 14:28:24 vs-mail systemd[1]: mariadb.service: Control process exited, code=exited, status=226/NAMESPACE
févr. 06 14:28:24 vs-mail systemd[1]: mariadb.service: Failed with result 'exit-code'.
févr. 06 14:28:24 vs-mail systemd[1]: Failed to start MariaDB 10.3.18 database server.

I'm going mad with this, I made dozens of requests on google and read a ton of LXC/apparmor/proxmox-related posts and still don't manage to deal with it.

Please help.

 

[oguz]

hi,

did you try enabling 'Nesting' feature for the container?

try running pct set CTID -features nesting=1 && pct reboot CTID and see if it works

 

[grobs]

In fact no, and it works! Thanks you very much for the quick reply!

I don't really understand what this setting allows (containers in containers?).
Is there any security risk with enabling this feature? And if no, why isn't it enabled by default?

 

[oguz]

i'm glad it works!

I don't really understand what this setting allows (containers in containers?).

yes. for example running docker containers in LXC (although we don't recommend this, some users want it)

Is there any security risk with enabling this feature? And if no, why isn't it enabled by default?

it exposes procfs and sysfs contents of the host to the guest, so there's some security implications (if your container gets hacked for example, they can access /proc and /sys on the host). that's why it's not enabled by default. however the latest containers sometimes need it for specific things to work (like in your case)

you can mark the thread as [SOLVED] so others know what to expect as well

 

[grobs]

Ok, I understand, thanks for those informations.
Is there any way to allow only namespace spawning rather than enabling the whole nesting feature?
The fact that the guest could have access to /proc and /sys on the host is pretty bad actually so it looks more like a workaround than a real solution.

 

[oguz]

Is there any way to allow only namespace spawning rather than enabling the whole nesting feature?

nesting is mount namespacing. unfortunately if you need this feature you have to enable it. some containers just need it at this point because of incompatibilities with systemd implementations/versions

 

[grobs]

Ok thanks.
I created an issu on the munin-monitoring's GitHub: https://github.com/munin-monitoring/munin/issues/1278

 

[souzahub]

hi,

did you try enabling 'Nesting' feature for the container?

try running pct set CTID -features nesting=1 && pct reboot CTID and see if it works

Thank you so much!

Everybody says.... "just enable nesting. Easy!" But nobody is kind enough to give a hint where or how to enable nesting.

I had no idea about pct set CTID -features nesting=1

Thank you for being helpful instead of show off.

 

[oguz]

Thank you so much!

Everybody says.... "just enable nesting. Easy!" But nobody is kind enough to give a hint where or how to enable nesting.

I had no idea about pct set CTID -features nesting=1

Thank you for being helpful instead of show off.

you can also do in the GUI: Container -> Features -> check Nesting

you're welcome