docker in lxc container

[limone]

hi guys,

i found this tutorial to run docker in lxc for proxmox: https://www.solaris-cookbook.eu/virtualisation/proxmox/proxmox-lxc-running-docker-inside-container/

basically just add

lxc.apparmor.profile: unconfined
lxc.cgroup.devices.allow: a
lxc.cap.drop:

to the containers config.

it works, only downside is that the filesystem used for docker is vfs. is this due to lxc, or because i've not installed something on the host node to use aufs or devicemapper?

 

[limone]

ok, seems like aufs works in lxc, so the question is: am i missing some packages on the proxmox node, or is it limited by proxmox itself?

http://ashish1099.github.io/blog/2015/05/23/docker-inside-lxc/

 

[atl]

It works! you can use aufs or overlay file system. For using it within a LXC container, you have to load the module on the proxmox host. I've inserted it into /etc/modules-load.d/modules.conf :

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
aufs
overlay

After reboot you can check, if the module was loaded:

root@proxmox:~# lsmod | grep -E 'overlay|aufs'
overlay                77824  8
aufs                  237568  0

Within the LXC container the same command should show, that the modules are loaded. Then change the docker configuration /etc/docker/daemon.json to use the wanted filesystem. I'm using overlay file system within LXC:

root@lxc:~# cat /etc/docker/daemon.json
{
  "storage-driver": "overlay2"
}

After restart docker service you can see with docker info, if it works:

root@lxc:~# docker info
Containers: 18
 Running: 0
 Paused: 0
 Stopped: 18
Images: 4
Server Version: 17.05.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
...

 

[LnxBil]

Yes, this has been discussed already in https://forum.proxmox.com/threads/container-linux-in-a-kvm.43136/#post-207264

... and you have a very, very insecure system (hypervisor, not container or VM) with this.

 

[jester]

Comment #4 might be interesting: https://discuss.linuxcontainers.org/t/run-docker-in-lxc-is-secure/1962/4

 

[kcallis]

I happened to run across this thread when I was looking around for the same topic. Since I am using pve on a laptop, I am always looking for ways to reduce resource usage. If I read this correctly, I could spin up a ubuntu 18.04 lxc container, and from there, install docker. I am trying to better understand docker and especially storage, my question is this: Assuming that I make the aforementioned container with 25G allocated, how does on create a docker container and make use of the storage outside of the container? For instance, if I were to spin up portainer, that is not an issue because it is relatively light on storage usage. But after this, I spin up postgreSQL and then create a 50G database, if my lxc container is only 25G, how would I make use of the storage on the host (pve with the thin-kvm or local) to store my database?

 

[morph027]

Why not directly installing docker on top of the pve host?

 

[kcallis]

I currently have docker sitting on the pve body. I decided to docker based version of atomic toolkit which is various media apps like plex, Sabnzbd, sonarr, etc. Lots of ports to keep track of, and more importantly, as I add more apps, there is more complexity. For instance, say I wanted to bring a docker image of WordPress. I would have to deal with changing port, etc.

I would rather do is create a lxc container called media (Ubuntu 18.04 image) and then docker based atomic toolkit and know to connect to http://media:5000 or http://media:32400 to get to Plex. Then I can spin up another lxc image and dockerize Nagios and other monitoring apps and get that running with minimal changes.

 

[LnxBil]

But after this, I spin up postgreSQL and then create a 50G database, if my lxc container is only 25G, how would I make use of the storage on the host (pve with the thin-kvm or local) to store my database?

You cannot.

If you're not so concerned about hypervisor security (you're on a laptop, so you normally don't), just run Docker on your host as @morph027 suggests. Best is to use Docker with ZFS so that you can have quota and separation on storage level.

 

[Flysquid]

I've been able to run docker in an LXC and have the docker container access the host storage. Whatever storage you want mounted to the host, you'd mount it to the LXC. Once the LXC can see the storage, you can start a docker container to access the mounted path. I do this for my docker container to see my nfs share.

As to the questions of why not just install docker on the host alongside PVE? Because I like the separation. The container can be backed up and migrated. The host can die or I tweak PVE incorrectly and have to reinstall. Being a home environment and no external access in, my concern for security is a minimal. I am hosting for me.

 

[rcd]

Having the same problem here. Ran the "modprobe overlay" on the host, verified presense in the container, but docker still fails with a not-very-helpful error message:

Oct 18 22:57:31 docker2 systemd[1]: Starting Docker Socket for the API.
Oct 18 22:57:31 docker2 systemd[1]: Listening on Docker Socket for the API.
Oct 18 22:57:31 docker2 systemd[1]: docker.service: Start request repeated too quickly.
Oct 18 22:57:31 docker2 systemd[1]: docker.service: Failed with result 'exit-code'.
Oct 18 22:57:31 docker2 systemd[1]: Failed to start Docker Application Container Engine.
Oct 18 22:57:31 docker2 systemd[1]: docker.socket: Failed with result 'service-start-limit-hit'.

The overlay driver is present.

root@docker2:~# lsmod | grep overlay
overlay               114688  0

If I remove the daemon.json with the overlay2 line it starts fine, but of course using vfs storage driver, which is super inefficient.
Any ideas?

 

[LnxBil]

Look with journalctl.

I recently installed Docker on a Debian LXC and it was very smooth without any hickups.