[SOLVED] Docker in LXC läuft nicht mehr

[H4R0]

Komisch, bei mir läuft noch alles mit "lxc-pve: 4.0.6-1" auch nach reboot. Soweit keine Probleme mit meinen beiden Docker-LXCs.

Du hattest mit sicherheit forwarding bereits aktiviert.

Ich gehe davon aus, dass es striktere apparmor regeln gibt im neuen Paket.

 

[Dunuin]

Du hattest mit sicherheit forwarding bereits aktiviert.

Ich gehe davon aus, dass es striktere apparmor regeln gibt im neuen Paket.

Ja, forwarding war aktiviert. Hatte gelesen das sollte man im Docker-Gast eh machen, damit da Docker ordentlich per iptables zwischen den Ports des Docker-Containers und den Ports des LXCs forwarden kann, damit die Docker-Container auch außerhalb des LXC erreichbar sind.
Mit deaktiviertem Forwarding und ohne entsprechende Iptable-Forwarding-Einträge waren auch vor Monaten schon meine Docker-Container nicht von außerhalb des LXCs erreichbar.

 

[miqu]

The downgrade via apt install lxc-pve:amd64=4.0.3-1 solved the issue for me on two lxc containers. The one was running docker and the other was the openvpn turnkey container which was also affected. Thanks!

 

[MinecraftiUndCo]

Wir hatten exakt das gleiche Problem bei unseren Containern... Danke, dass es dieses Forum hier gibt, sonst hätte ich wahrscheinlich noch die nächsten 5 Jahre nach der Lösung gesucht. Habe zuerst gedacht, ich hätte irgendwas an Docker zerstört und habe da alles kaputt gemcht gehabt. xD

Wäre schön, wenn dieses Problem in einer der nächsten lxc-pve Versionen behoben werden könnte und man nicht mit dem Downgrade arbeiten müsste.

 

[t.lamprecht]

Auf pvetest gibt es jetzt eine aktualisiertes lxc-pve Paket mit Version 4.0.6-2 , damit sollten die Probleme behoben sein.

Entweder pvetest Repository einbinden:
https://pve.proxmox.com/wiki/Package_Repositories#sysadmin_test_repo

Oder direkt Download + Installieren

wget http://download.proxmox.com/debian/pve/dists/buster/pvetest/binary-amd64/lxc-pve_4.0.6-2_amd64.deb
# Prüfsumme kontrollieren
sha256sum lxc-pve_4.0.6-2_amd64.deb
  894f07f6839bd322d74f778062732d36a36643667c33c4466796f883f7dbb291  lxc-pve_4.0.6-2_amd64.deb
apt install ./lxc-pve_4.0.6-2_amd64.deb

 

[glade]

The problem is not yet completely solved by the update. Although the kernel options like ip_forward can now be set again, services like dhrelay do not recognize the interfaces:

Feb 9 10:13:40 router dhcrelay[92]: Discarding packet received on ethnet1 interface that has no IPv4 address assigned.
Feb 9 10:13:45 router dhcrelay[92]: Discarding packet received on ethnet2 interface that has no IPv4 address assigned.
Feb 9 10:14:00 router dhcrelay[92]: Discarding packet received on ethnet2 interface that has no IPv4 address assigned.
Feb 9 10:14:45 router dhcrelay[92]: Discarding packet received on ethnet3 interface that has no IPv4 address assigned.
Feb 9 10:15:49 router dhcrelay[92]: Discarding packet received on ethnet3 interface that has no IPv4 address assigned.
Feb 9 10:16:54 router dhcrelay[92]: Discarding packet received on ethnet3 interface that has no IPv4 address assigned.

All interfaces have bound IP addresses. Access also works if the IP addresses are manually bound in the networks.

Best regards
Robert

 

[t.lamprecht]

services like dhrelay do not recognize the interfaces:

Can you confirm that this use case also just works if you downgrade to version 4.0.3-1?

 

[glade]

Hello Thomas,

yes it works with the downgrade. It also works with version 4.0.6-2 if you configure a delay. I have helped myself with a delay of 30 seconds:

/etc/systemd/system/isc-dhcp-relay.service.d/override.conf

[Service]
ExecStartPre=/bin/sleep 30


From my side, the problem would be solved. But maybe the whole thing has other effects.

By the way, I noticed that the network in the unchanged configuration is apparently not yet ready when starting the DHCRELAY.

syslog:
...
Feb 9 10:23:38 router dhcrelay[100]: receive_packet failed on eth0: Network is down
Feb 9 10:23:38 router dhcrelay[100]: receive_packet failed on ethnet0: Network is down
Feb 9 10:23:38 router systemd[1]: Started LSB: DHCP relay.
Feb 9 10:23:38 router dhcrelay[100]: receive_packet failed on ethet1: Network is down
Feb 9 10:23:38 router dhcrelay[100]: receive_packet failed on ethet2: Network is down
Feb 9 10:23:38 router dhcrelay[100]: receive_packet failed on ethet3: Network is down
...

Best regards
Robert

 

[Benr]

Hi, I have the same problem after a power failure in my house that you can see as a reboot. I think I updated Proxmos earlier without rebooting so all my LXC Containers with Docker in it dont work anymore after I booted up my Proxmox Server.

I am on PVE 7.1-10
when i type apt show lxc-pve
I get

root@proxmox:~# apt show lxc-pve
Package: lxc-pve
Version: 4.0.11-1
Priority: optional
Section: admin
Maintainer: Proxmox Support Team <support@proxmox.com>
Installed-Size: 23.8 MB
Provides: liblxc1, lxc
Depends: apparmor, bridge-utils, criu (>= 1.5.2-1), libcap2 (>= 1:2.10), libgnutlsxx28, lxcfs, python3, uidmap, libc6 (>= 2.27), libgcc-s1 (>= 3.3.1), libseccomp2 (>= 2.5.0)
Conflicts: liblxc1, lxc
Breaks: pve-container (<< 3.1-1)
Replaces: liblxc1, lxc
Homepage: https://linuxcontainers.org
Download-Size: 2,339 kB
APT-Manual-Installed: yes
APT-Sources: http://download.proxmox.com/debian/pve bullseye/pve-no-subscription amd64 Packages

when I try to downgrade with apt install lxc-pve:amd64=4.0.3-1 as suggested multiple times, it says

root@proxmox:~# apt install lxc-pve:amd64=4.0.3-1
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Version '4.0.3-1' for 'lxc-pve' was not found

what can I do?
Maybe @t.lamprecht you have any idea? I have 90% of my Docker Containers in a VM but the most important (Bitwarden) is a non working Proxmox LXC

thanks in before for any help

__________________________________________________

EDIT
I was able to revive my docker in an ubuntu 20.04 LXC with a simple apt update && apt upgrade inside the LXC Container.
With my second LXC, a debian 10 container, i wasnt that lucky sadly. I would appreciate any help.

 

[juliokele]

Hi, I have the same problem after a power failure in my house that you can see as a reboot. I think I updated Proxmos earlier without rebooting so all my LXC Containers with Docker in it dont work anymore after I booted up my Proxmox Server.

I am on PVE 7.1-10
when i type apt show lxc-pve
I get

root@proxmox:~# apt show lxc-pve
Package: lxc-pve
Version: 4.0.11-1
Priority: optional
Section: admin
Maintainer: Proxmox Support Team <support@proxmox.com>
Installed-Size: 23.8 MB
Provides: liblxc1, lxc
Depends: apparmor, bridge-utils, criu (>= 1.5.2-1), libcap2 (>= 1:2.10), libgnutlsxx28, lxcfs, python3, uidmap, libc6 (>= 2.27), libgcc-s1 (>= 3.3.1), libseccomp2 (>= 2.5.0)
Conflicts: liblxc1, lxc
Breaks: pve-container (<< 3.1-1)
Replaces: liblxc1, lxc
Homepage: https://linuxcontainers.org
Download-Size: 2,339 kB
APT-Manual-Installed: yes
APT-Sources: http://download.proxmox.com/debian/pve bullseye/pve-no-subscription amd64 Packages

when I try to downgrade with apt install lxc-pve:amd64=4.0.3-1 as suggested multiple times, it says

root@proxmox:~# apt install lxc-pve:amd64=4.0.3-1
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Version '4.0.3-1' for 'lxc-pve' was not found

what can I do?
Maybe @t.lamprecht you have any idea? I have 90% of my Docker Containers in a VM but the most important (Bitwarden) is a non working Proxmox LXC

thanks in before for any help

__________________________________________________

EDIT
I was able to revive my docker in an ubuntu 20.04 LXC with a simple apt update && apt upgrade inside the LXC Container.
With my second LXC, a debian 10 container, i wasnt that lucky sadly. I would appreciate any help.

try your debian container (if not already set) with Features: nesting=on

 

[t.lamprecht]

have the same problem after a power failure in my house that you can see as a reboot

That could have other side effects as well, depending from if you have a UPS or enterprise disk that can cleanly shutdown on a power failure.

I think I updated Proxmos earlier without rebooting

From what version did you upgrade earlier? iow. was this an major upgrade or was the system already cleanly on 7.x before the upgrade?

 

[Benr]

try your debian container (if not already set) with Features: nesting=on

thanks for the hint, but this is already checked:

From what version did you upgrade earlier? iow. was this an major upgrade or was the system already cleanly on 7.x before the upgrade?

I could have sworn I installed a 7.x last year but i looked for the isos I downloaded and only could find a 6.3-1.iso from march 21 so I think that was the version before the upgrade




here are some commands i tried, I dont know if they help but apparently the docker socket from the host cant be found.

root@Reverse-Proxy:~# docker ps
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

root@Reverse-Proxy:~# apt upgrade
[...]
Do you want to continue? [Y/n]
Setting up docker.io (18.09.1+dfsg1-7.1+deb10u3) ...
Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.
invoke-rc.d: initscript docker, action "start" failed.
* docker.service - Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Mon 2022-01-31 06:42:22 UTC; 9ms ago
TriggeredBy: * docker.socket
       Docs: https://docs.docker.com
    Process: 1616 ExecStart=/usr/sbin/dockerd -H fd:// $DOCKER_OPTS (code=exited, status=1/FAILURE)
   Main PID: 1616 (code=exited, status=1/FAILURE)
        CPU: 136ms
dpkg: error processing package docker.io (--configure):
 installed docker.io package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 docker.io
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@Reverse-Proxy:~#

 

[t.lamprecht]

I could have sworn I installed a 7.x last year but i looked for the isos I downloaded and only could find a 6.3-1.iso from march 21 so I think that was the version before the upgrade

You can check /var/log/apt/history.log (and rotated versions) about your update history. In any case, Proxmox VE requires active admin steps to upgrade between major versions, so did you recently change the repositories from buster to bullseye or the like? As else apt won't pull you from 6.x to 7.x through a standard package upgrade.
https://pve.proxmox.com/wiki/Upgrade_from_6.x_to_7.0

Setting up docker.io (18.09.1+dfsg1-7.1+deb10u3) ...

The main issue seems to be your quite old docker version, another user reported that it works out fine with docker-ce 20.10:
https://forum.proxmox.com/threads/l...rade-from-6-4-to-7-0.92034/page-2#post-444744

 

[Benr]

The main issue seems to be your quite old docker version

Holy freaking cow, i reinstalled docker in the container and it revived the old docker containers! Yeeeha!!
Thanks so much for your help, i love proxmox and this forum, you made my week!

for others, that have similar issues:
i simply followed the installation guide:
https://docs.docker.com/engine/install/debian/

root@Reverse-Proxy:~# apt remove docker docker-engine docker.io containerd runc

root@Reverse-Proxy:~# apt-get install     ca-certificates     curl     gnupg     lsb-release

root@Reverse-Proxy:~#  curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

root@Reverse-Proxy:~# echo   "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

root@Reverse-Proxy:~# apt update

root@Reverse-Proxy:~# apt install docker-ce docker-ce-cli containerd.io


-->
root@Reverse-Proxy:~# docker ps
CONTAINER ID   IMAGE                             COMMAND                  CREATED         STATUS                   PORTS                                                                                  NAMES
ec6484ee28c8   jc21/nginx-proxy-manager:latest   "/init"                  9 months ago    Up 5 minutes (healthy)   0.0.0.0:80-81->80-81/tcp, :::80-81->80-81/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   dcompose_reverseproxy_1
0fc778386e05   mariadb:latest                    "docker-entrypoint.s…"   9 months ago    Up 5 minutes             0.0.0.0:3306->3306/tcp, :::3306->3306/tcp                                              dcompose_db_1
23683fd05dc5   portainer/portainer-ce            "/portainer"             12 months ago   Up 5 minutes             0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp   portainer-ce
root@Reverse-Proxy:~#

thank you again Thomas, have a good day