LXC roadmap ?

[dipe]

Being reasonably happy with a newly installed 4.1 cluster I'd be interested what the plans and priorities are for LXC for the next PVE release. We don't seem to have the issues others reported (e.g. snapshots / backups ---- we use local ZFS) and saw only one smallish bug so far https://bugzilla.proxmox.com/show_bug.cgi?id=890

We are extremely interested in getting live migration of containers with local storage to work again like it did with openvz. I was wondering if this is targeted for the next release but also how it is going to be implemented. Docker as well as Ubuntu lxd seem to use CRIU for this. Is proxmox planning to do the same?

The next big thing on our list is nested containers. (docker (or lxc) containers inside lxc containers) . I saw somewhere that the LXD folks want to support this but could not find a hint that this is already working. (there has been an open bug for a while https://bugzilla.proxmox.com/show_bug.cgi?id=672 )

Is there any more info you can share on your plans with lxc?

dp

 

[dietmar]

Docker as well as Ubuntu lxd seem to use CRIU for this. Is proxmox planning to do the same?

Yes, we already ship CRIU packages, but they are not stable enough for now. But yes, we plan to do it with CRIU.

The next big thing on our list is nested containers. (docker (or lxc) containers inside lxc containers) .

This is low priority on my list, but we will support that if LXC can do it.

Is there any more info you can share on your plans with lxc?

We currently make mountpoint support more mature, improve vzdump, implemented user quota support and read-only mounts, ...

 

[dipe]

Thanks Dietmar, is CRIU planned for the next release 4.2 or is it more a later release 4.3 or 4.4 ?

 

[dietmar]

I think 4.3 is more realistic ...

 

[dipe]

The next big thing on our list is nested containers. (docker (or lxc) containers inside lxc containers) . I saw somewhere that the LXD folks want to support this but could not find a hint that this is already working. (there has been an open bug for a while https://bugzilla.proxmox.com/show_bug.cgi?id=672 )
dp

Just to follow up on my posting from a few months ago:

The LXD folks now have nested containers working (docker or lxc inside lxc)

https://www.stgraber.org/2016/04/13/lxd-2-0-docker-in-lxd-712/

https://insights.ubuntu.com/2016/04/15/lxd-2-0-lxd-in-lxd-812/

There was a dependency on cgroup namespaces which is now in Kernel 4.6 (it was backported to Kernel 4.4 for Ubuntu 16.04). There are a number of other dependencies though .... but it could be interesting if you are planning on Kernel 4.6 for Proxmox 4.3 or 4.4 anyway

Also here is an interesting option for running a whole docker swarm inside lxc http://andrea.corbellini.name/2016/04/13/docker-swarm-inside-lxc/ , can be pretty useful if one wanted to simulate a large cluster

 

[dietmar]

There was a dependency on cgroup namespaces which is now in Kernel 4.6 (it was backported to Kernel 4.4 for Ubuntu 16.04).

The current 4.4 kernel is a LTS kernel, so we plan to keep that for a while. But everything required for docker inside LXC is there. I am not sure if new cgroup interface works on debian jessie, but we will test that option.

 

[dietmar]

Also here is an interesting option for running a whole docker swarm inside lxc

Besides, it is perfectly valid to run docker and docker swarm inside a VM.

 

[harvie]

I've just tried to run docker 18.09.1 from Debian 10 in LXC CT on PVE. I was able to install the portainer on docker in LXC, everything works, i can open the portainer web ui. But then i tried to spin up the docker swarm with portainer and it seems that portainer runs, but the ports are not forwarded to the container, so i can't access the portainer web ui.

This is probably down to the fact that docker swarm uses bit more complex iptables setup than plain docker, so i think this might be possible to figure out if proxmox team carefuly tunes the iptables support in LXC to reflect the docker swarm needs.

 

[MadalinC]

I've just tried to run docker 18.09.1 from Debian 10 in LXC CT on PVE. I was able to install the portainer on docker in LXC, everything works, i can open the portainer web ui. But then i tried to spin up the docker swarm with portainer and it seems that portainer runs, but the ports are not forwarded to the container, so i can't access the portainer web ui.

This is probably down to the fact that docker swarm uses bit more complex iptables setup than plain docker, so i think this might be possible to figure out if proxmox team carefuly tunes the iptables support in LXC to reflect the docker swarm needs.

This is pretty much the same problem I have now with my setup. Swarm cannot publish ports for some reason and I couldn't make it work yet.

Did you have any luck with your setup or did you just go with VMs and be done with it?

Thanks!