[px5] new CT fail to start: mknod: …/rootfs/dev/rbd3: Operation not permitted

[grin]

[proxmox5]
Newly created unprivileged lxc container fails to start. The failure is rather ugly, since there is basically no info on it:

Aug 16 00:25:25 elton lxc-start[39248]: lxc-start: tools/lxc_start.c: main: 366 The container failed to start.
Aug 16 00:25:25 elton lxc-start[39248]: lxc-start: tools/lxc_start.c: main: 368 To get more details, run the container in foreground mode.
Aug 16 00:25:25 elton lxc-start[39248]: lxc-start: tools/lxc_start.c: main: 370 Additional information can be obtained by setting the --logfile and --logpriority options.​

This usually sucks. As it turns out it sucks more: while the boot fails in lxc-pve-autodev-hook the script fails to get its warnings or errors logged, or if they're logged I'm not aware where.

Patching the file with manual logging turns out that it tries to run_command and fail:

mknod: /usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/rbd3: Operation not permitted​

(as you see, this is a ceph-rbd backed CT)
I do not have the patience to patch multiple files (due to missing log) to figure out why it's not permitted; as a temporary measure I eval{}ed the command and discarded the error status, which isn't nice but works, for now.

 

[fabian]

LXC logging is a bit cumbersome unfortunately, but the PVE Admin guide tells you how to start a container to get debug logs

 

[grin]

LXC logging is a bit cumbersome unfortunately, but the PVE Admin guide tells you how to start a container to get debug logs

Ah, damn, foreground puts the output on the console, but background doesn't capture them in the logfile. Stooopid!
Thanks!

Seems like the container is missing CAP_MKNOD, and the systemd (be it damned in the fires of hell forever) autodev feature is not used. That's a quite serious bug: no newly created unprivileged container is able to run anymore.

 

[fabian]

starting an unprivileged container using KRBD works just fine here - if you think there is a bug please post the output of pveversion -v, ceph --version, the container and storage configs and the debug log.

 

[grin]

Okay, let's discuss this over there.
Thanks.