Rootless Docker inside unprivileged LXC container

[Firm]

Hi,

Has anybody succeeded with running rootless (as ordinal user not root) Docker inside unprivileged LXC container? I followed this official guide: https://docs.docker.com/engine/security/rootless/. Installation failed with the following message:

dockerd-rootless.sh[355]: [rootlesskit:parent] error: failed to setup UID/GID map: newuidmap 366 [0 1000 1 1 100000 65536] failed: newuidmap: write to uid_map failed: Operation not permitted

I've checked this article: https://ubuntu.com/blog/nested-containers-in-lxd and applied necessary changes both to host (Proxmox) and container-based /etc/subuid, /etc/subgid files but this didn't help much:

dockerd-rootless.sh[928]: [rootlesskit:parent] error: failed to setup UID/GID map: newuidmap 939 [0 1000 1 1 65536 131072] failed: newuidmap: write to uid_map failed: Operation not permitted

Even strace-ing brought nothing new: code fails on write() operation to uid_map file.

Proxmox:

# pveversion
pve-manager/6.4-5/6c7bf5de (running kernel: 5.4.106-1-pve)

LXC container:

Ubuntu 20.04.2 LTS (GNU/Linux 5.4.106-1-pve x86_64)

Are there any other things I could miss?

P.S. Nesting is enabled for container.

Regards,
Alex

 

[Stefan_R]

Even strace-ing brought nothing new: code fails on write() operation to uid_map file.

If this is running inside your container, the permission checks would fail accessing the files inside that container too, i.e. check your uid_map file in your container, not on the host.

I'm curious though - why do you want to run rootless docker inside an unprivileged container? That's just doing the isolation work twice, the container is already in a user namespace, so root in container doesn't equal root on host, and as such can't do much...

 

[AnK]

Hi,
I know it's been a long time ago, but with that post, I managed to get rootless podman working in a Alma9 LXC container:

https://www.reddit.com/r/podman/comments/t111i1/running_podman_in_an_lxc_container_security/

 

[luison]

Hi, wondering if anyone had any luck with this setup or can point to an up-to-date guide to get ideally unprivileged and rootless docker. Alternatively, I'll just have to consider one or both or maintain our current docker on KVM if I can get a proper way to mount and share host directories.

I have not succeeded in my tries, including adding the configs on the previous entry in the container and activating nesting and fuse.

I keep hitting the error:

error: failed to setup UID/GID map: newuidmap 3994 [0 1100 1 1 10000 65536] failed: newuidmap: write to uid_map failed: Operation not permitted

when executing the rootless docker command.

Any updates on how to have a docker server as secure as possible on a CT would be appreciated.

 

[LnxBil]

Alternatively, I'll just have to consider one or both or maintain our current docker on KVM if I can get a proper way to mount and share host directories.

Just virtualize your storage, so that it is also HA and then mount the storage from there in your Docker compute nodes. You should always stay on the same layer (PaaS and St(orage)aaS) on PVE as IaaS.

 

[kegham]

Have you managed a way to use it rootless podman in LXC?

 

[luison]

Rootless docker causes us an additional limitation regarding having a full firewall working as we wanted. The docker0 network becomes unavailable to root, so we've stopped testing it under unprivileged LXC for the moment, so can not provide any other info, sorry.

 

[cpcheng]

Hi, wondering if anyone had any luck with this setup or can point to an up-to-date guide to get ideally unprivileged and rootless docker. Alternatively, I'll just have to consider one or both or maintain our current docker on KVM if I can get a proper way to mount and share host directories.

I have not succeeded in my tries, including adding the configs on the previous entry in the container and activating nesting and fuse.

I keep hitting the error:

error: failed to setup UID/GID map: newuidmap 3994 [0 1100 1 1 10000 65536] failed: newuidmap: write to uid_map failed: Operation not permitted

when executing the rootless docker command.

Any updates on how to have a docker server as secure as possible on a CT would be appreciated.

For reference, you need to increase the UID and GID available to the LXC. This is done via adjusting the /etc/subuid and /etc/subgid of Proxmox. For example, I changed the default 100000:65536 to 100000:165536 then on the LXC, you need to map with:

lxc.idmap: u 0 100000 165536
lxc.idmap: g 0 100000 165536

However, I'm having trouble getting docker to run, and it constantly errors out with "medium not found". How were you able to get past that?

EDIT: Needed to allow TUN
lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file

Docker rootless works awesome for me now.