[SOLVED] LXC + docker, containerd cannot find /lib/modules/4.15.18-9-pve

Discussion in 'Proxmox VE: Installation and configuration' started by Klox, Dec 3, 2018.

Tags:
  1. Klox

    Klox New Member

    Joined:
    Dec 3, 2018
    Messages:
    2
    Likes Received:
    0
    Hi all,

    Been stuck on this for quite some time and looking for some feedback. I'm running Proxmox 5.2 and I currently have 6 LXC Ubuntu 18.10 containers. 1 hosts rancher, 5 others host k8s worker/etcd/controller machines. I've been using the unsecure mode outlined here

    http-s://ww-w.solaris-cookbook.eu/virtualisation/proxmox/proxmox-lxc-running-docker-inside-container/ (had to break the link due to being a new forum user)

    to change apparmor etc. settings. This has worked for a month, including after updating the container's packages. I am now trying to add a 6th container and I run into an issue running docker:

    This led me to seeing containerd failing to startup:

    I am installing docker-ce from docker's repository. In my /lib/modules for proxmox I of course have 4.15.18-9-pve, but in the LXC container I only have 4.18.0-12-lowlatency. I saw that this was actually installed during the installation of docker-ce, dependency "linux-image-unsigned-4.18.0-12-lowlatency" and "linux-modules-4.18.0-12-lowlatency". In the containers that are working I have "4.18.0-1003-kvm".

    So I tried installing the different package:

    Code:
    apt install linux-image-4.18.0-1003-kvm linux-modules-4.18.0-1003-kvm
    and the "4.18.0-1003-kvm" folder is available in /lib/modules, but containerd is still looking for "4.15.18-9-pve".

    pveversion: pve-manager/5.2-12/ba196e4b (running kernel: 4.15.18-9-pve)
    docker-ce: :18.09.0~3-0~ubuntu-bionic
    LXC template: ubuntu-18.10-standard_18.10-1_amd64.tar.gz
    LXC's uname: Linux docker-test2 4.15.18-9-pve #1 SMP PVE 4.15.18-30 (Thu, 15 Nov 2018 13:32:46 +0100) x86_64 x86_64 x86_64 GNU/Linux

    Proxmox /etc/pve/lxc/110.conf:

    Any insight would be great; I am not seeing how my process is any different from a month ago and I took pretty good notes. I am still splunking around.

    Thanks!
     
  2. morph027

    morph027 Active Member

    Joined:
    Mar 22, 2013
    Messages:
    408
    Likes Received:
    49
    Try;

    Code:
    pct set xxx --mp0 /lib/modules/$(uname -r),mp=/lib/modules/$(uname -r),ro=1
    
     
    Klox likes this.
  3. Klox

    Klox New Member

    Joined:
    Dec 3, 2018
    Messages:
    2
    Likes Received:
    0
    Magical, thank you! I had created two test containers both experiencing the same problem, and doing this on one ALSO fixed it in the second. I am not sure what to make of that. This seems to stick over container reboots too. Thanks!
     
  4. morph027

    morph027 Active Member

    Joined:
    Mar 22, 2013
    Messages:
    408
    Likes Received:
    49
    Basically your containers are running the host kernel, therefore we're just bind-mounting the host kernel modules into the container (readonly). You'll need to repeat this step after an pve host kernel update.
     
  5. der-brumm-baer

    der-brumm-baer New Member

    Joined:
    Aug 22, 2018
    Messages:
    6
    Likes Received:
    1
    Hello,

    i run into the same problem. But i find a different solution which not depends on the host kernel.

    There is a service called containerd which starts before docker
    this service tries to modprobe the overlay Modul


    Code:
    cat /lib/systemd/system/containerd.service
    
    Code:
    [Unit]
    Description=containerd container runtime
    Documentation=htxxps://containerd.io
    After=network.target
    
    [Service]
    ExecStartPre=/sbin/modprobe overlay
    ExecStart=/usr/bin/containerd
    KillMode=process
    Delegate=yes
    LimitNOFILE=1048576
    # Having non-zero Limit*s causes performance problems due to accounting overhead
    # in the kernel. We recommend using cgroups to do container-local accounting.
    LimitNPROC=infinity
    LimitCORE=infinity
    TasksMax=infinity
    
    [Install]
    WantedBy=multi-user.target
    
    so in the LXC container you can create a file
    Code:
    nano /etc/systemd/system/containerd.service.d/override.conf
    
    with content:
    Code:
    [Service]
    ExecStartPre=
    ExecStartPre=
    
    do
    Code:
    systemctl daemon-reload
    service docker restart
    
    and the docker service will be running
     
    wasp likes this.
  6. der-brumm-baer

    der-brumm-baer New Member

    Joined:
    Aug 22, 2018
    Messages:
    6
    Likes Received:
    1
    Hello,

    I have another Problem with the actual Docker Version:

    The new Version of Docker CE in Debian Stretch

    18.09.0~3-0~debian-stretch

    which has the new containerd Service, keeps the LXC from doing a complete shutdown.

    When I shutdown the LXC Container it ends up in an undefined state. Not really stopped and not really running.
    It keeps the filesystems mounted.
    I had to completley shutdown the host to get rid of this.

    But it is not only affecting LXC Containers, also KVM-VM won't shutdown completely.

    I tested the previous Version of Docker CE

    18.06.1~ce~3-0~debian

    in a KVM-VM and it did a complete shutdown
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice