5.3: Docker on LXC on ZFS

Discussion in 'Proxmox VE: Installation and configuration' started by koalillo, Dec 5, 2018.

Tags:
  1. koalillo

    koalillo New Member

    Joined:
    Nov 1, 2018
    Messages:
    9
    Likes Received:
    0
    Hi,

    I'm toying with running Docker inside a CentOS 7 LXC on a Proxmox whose storage is ZFS. The container has:

    $ cat /etc/pve/lxc/209.conf
    ...
    features: nesting=1
    ...

    and is a privileged container.

    For starters, the only storage driver that allows me to start the Docker service is vfs. ZFS is not supported OOB (I suppose some tinkering might help, but...).

    Starting with VFS, starting a container fails:

    $ sudo docker run -it --rm centos
    ...
    /usr/bin/docker-current: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:327: setting cgroup config for procHooks process caused \"failed to write c 10:200 rwm to devices.allow: write /sys/fs/cgroup/devices/system.slice/docker-29dec37417e09a1b8e118bd9e7402578307c8c73ad562ee887f37efa7bc85449.scope/devices.allow: operation not permitted\"".
    ERRO[0016] error getting events from daemon: context canceled
     
  2. wbumiller

    wbumiller Proxmox Staff Member
    Staff Member

    Joined:
    Jun 23, 2015
    Messages:
    631
    Likes Received:
    73
    With privileged containers device usage is very restricted.
    If you need a privileged container you'll need to allow the use of /dev/net/tun (the c:10:200 it's trying to allow access to, used for tunnel and tap device setup):
    Code:
    # /etc/pve/lxc/$vmid.conf
    (...)
    lxc.cgroup.devices.allow = c 10:200 rwm
    With an unprivileged centos7 container with `features: nesting=1,keyctl=1` and docker-ce I can run the above `docker run` command you mentioned without any extra lines in the config.

    I'm not sure why it would fail on `c:10:200` which is /dev/net/tun though - the device node is not available on the unprivileged container by default either - however, the cgroup does permit access to it in that case, so the operation you posted above wouldn't error either.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. koalillo

    koalillo New Member

    Joined:
    Nov 1, 2018
    Messages:
    9
    Likes Received:
    0
    OK, using an unprivileged container helps. With keyctl I can run Docker. The overlay storage driver detects its on ZFS and refuses to work, but VFS works. I need to play with this as I suspect VFS is terrible.

    However, switching to an unprivileged container breaks my FreeIPA client setup.
     
  4. koalillo

    koalillo New Member

    Joined:
    Nov 1, 2018
    Messages:
    9
    Likes Received:
    0
  5. morph027

    morph027 Active Member

    Joined:
    Mar 22, 2013
    Messages:
    398
    Likes Received:
    46
    Not tried yet ... overlay2 probably?
     
  6. koalillo

    koalillo New Member

    Joined:
    Nov 1, 2018
    Messages:
    9
    Likes Received:
    0
    overlay2 detects you are in a COW filesystem and rejects to work.

    The zfs driver doesn't seem to work
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice