no connection to the internet from container

T

t_b

New Member
Nov 4, 2015
22
2
1
Hello,

I currenty facing network problems with the virtual adapter for the container network.
Here is my network config

/etc/network/interfaces
Code: 
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address 5.35.x.y
        netmask 255.255.255.0
        gateway 5.35.x.y

auto vmbr0
iface vmbr0 inet static
        address  10.0.0.1
        netmask  255.255.255.0
        bridge_ports none
        bridge_stp off
        bridge_fd 0

/etc/sysctl.conf
Code: 
net.ipv4.ip_forward=1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0

ip addr (proxmox)
Code: 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:1c:42:74:61:bd brd ff:ff:ff:ff:ff:ff
    inet 5.35.x.y/24 brd 5.35.x.y scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::21c:x:x:x/64 scope link 
       valid_lft forever preferred_lft forever
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fe:30:d6:f5:bd:91 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.1/24 brd 10.0.0.255 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::e44e:x:x:x/64 scope link 
       valid_lft forever preferred_lft forever
6: veth100i0@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether fe:30:d6:f5:bd:91 brd ff:ff:ff:ff:ff:ff link-netnsid 0

ping (proxmox)
Code: 
root@proxmox:~# ping 10.0.0.2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_seq=1 ttl=64 time=0.094 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=64 time=0.068 ms
^C
--- 10.0.0.2 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.068/0.081/0.094/0.013 ms
root@proxmox:~# ping google.com
PING google.com (172.217.23.174) 56(84) bytes of data.
64 bytes from fra15s22-in-f174.1e100.net (172.217.23.174): icmp_seq=1 ttl=56 time=3.84 ms
64 bytes from fra15s22-in-f174.1e100.net (172.217.23.174): icmp_seq=2 ttl=56 time=4.02 ms


ip addr (container)
Code: 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
7: eth0@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 6e:61:4d:af:d6:2c brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.2/24 brd 10.0.0.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::6c61:4dff:feaf:d62c/64 scope link 
       valid_lft forever preferred_lft forever

ping (container)
Code: 
root@container:~# ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.196 ms
64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.072 ms
^C
--- 10.0.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.072/0.134/0.196/0.062 ms
root@container:~# ping google.com
^C

I Thought it's a firewall problem so i disabled firewall on cluster & node but it made no difference. But in case here is the firewall config:

/etc/pve/firewall/cluster.fw
Code: 
[OPTIONS]

policy_in: ACCEPT
enable: 1

[RULES]

GROUP proxmox
IN ACCEPT -p tcp -dport 22 # ssh
IN DROP

[group proxmox]

IN ACCEPT -p tcp -dport 22 # ssh
IN ACCEPT -p tcp -dport 443 # https
IN ACCEPT -p tcp -dport 80 # http
 
your config looks OK to me
simple question, did you setup a DNS resolver in the container ?
ie, what is the output of /etc/resolv.conf ?
 
I've used the dns tab on the ui which generated following content in the container
Code: 
# --- BEGIN PVE ---
domain.tld
nameserver 80.237.x.y
nameserver 80.237.x.z
# --- END PVE ---

This are the nameserver from the hosting provider
 
manu said:
It seems you miss the masquerading bits in /etc/network/interfaces

see https://pve.proxmox.com/wiki/Network_Model, and look for "masquerade" in that page
Click to expand...

Hi,

I have a same issue.

I had no internet connection if i enable the firewall on the network card on my container. the container use my proxmox-host as gateway, to connect to the internet. if i enable the firewall on my container i have no connection. i can't ping also the proxmox host. the container have no public ip-adress.

more information you find in the post.

https://forum.proxmox.com/threads/proxmox-firewall-problem-container-bridges.33166/#post-163245


see also my attached screenshot.
Best regards,

Oliver
 

Attachments

  • container1.png
    container1.png
    25.7 KB · Views: 119
@manu
I've already tried that, but i've updated the vmbr0 section again with

Code: 
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth0 -j MASQUERADE

I restart the network with
Code: 
systemctl restart networking

also i stopped an started the container... same issue

EDIT:

What i not mentioned is that i'm using docker on the proxmox host. Here is the outut of iptables-save:

Code: 
# Generated by iptables-save v1.4.21 on Mon Mar  6 12:36:25 2017
*filter
:INPUT ACCEPT [197103:61219441]
:FORWARD ACCEPT [1:56]
:OUTPUT ACCEPT [199328:66915477]
:DOCKER - [0:0]
:DOCKER-ISOLATION - [0:0]
-A FORWARD -j DOCKER-ISOLATION
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 9000 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8765 -j ACCEPT
-A DOCKER -d 172.17.0.4/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8080 -j ACCEPT
-A DOCKER-ISOLATION -j RETURN
COMMIT
# Completed on Mon Mar  6 12:36:25 2017
# Generated by iptables-save v1.4.21 on Mon Mar  6 12:36:25 2017
*nat
:PREROUTING ACCEPT [22:1607]
:INPUT ACCEPT [21:1551]
:OUTPUT ACCEPT [252:15240]
:POSTROUTING ACCEPT [252:15240]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.17.0.2/32 -d 172.17.0.2/32 -p tcp -m tcp --dport 9000 -j MASQUERADE
-A POSTROUTING -s 172.17.0.3/32 -d 172.17.0.3/32 -p tcp -m tcp --dport 8765 -j MASQUERADE
-A POSTROUTING -s 172.17.0.4/32 -d 172.17.0.4/32 -p tcp -m tcp --dport 8080 -j MASQUERADE
-A POSTROUTING -s 10.0.0.0/24 -o eth0 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
-A DOCKER -d 127.0.0.1/32 ! -i docker0 -p tcp -m tcp --dport 9000 -j DNAT --to-destination 172.17.0.2:9000
-A DOCKER -d 127.0.0.1/32 ! -i docker0 -p tcp -m tcp --dport 8765 -j DNAT --to-destination 172.17.0.3:8765
-A DOCKER -d 127.0.0.1/32 ! -i docker0 -p tcp -m tcp --dport 8080 -j DNAT --to-destination 172.17.0.4:8080
COMMIT
# Completed on Mon Mar  6 12:36:25 2017
 
@manu You're right, that would be better, but since it's already a "virtualized root server" i'm not abled to create KVM machines, only containers. On the other hand @OliverB seem not having docker and running into the same issues. I'm willing to test remove the docker installation restart the complete environment and re-check the network connection. If it's not working it's probably not a docker issue.
 
@manu So, docker is away until the problems solved - since docker seems not to be the problem. I tried a lot and nothing helped. i compared the configuration with this of my personal instance with no luck.
 
@OliverB BTW, i'm getting exactly the same responses:


Code: 
22:31:00.897158 6e:61:4d:af:d6:2c > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.10.10.1 tell 10.10.10.2, length 28
22:31:00.897248 fe:1a:a1:78:1a:2e > 6e:61:4d:af:d6:2c, ethertype ARP (0x0806), length 42: Reply 10.10.10.1 is-at fe:1a:a1:78:1a:2e, length 28
22:31:00.897274 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.48951 > 80.237.x.y.53: 54093+ A? google.com. (28)
22:31:05.902463 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.53921 > 80.237.x.y.53: 54093+ A? google.com. (28)
22:31:10.908382 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.48951 > 80.237.x.z.53: 54093+ A? google.com. (28)
22:31:18.270979 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.36427 > 80.237.x.z.53: 52448+ A? google.com. (28)
22:32:30.101175 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.39268 > 80.237.x.z.53: 22787+ A? google.com. (28)
 
t_b said:
@OliverB BTW, i'm getting exactly the same responses:


Code: 
22:31:00.897158 6e:61:4d:af:d6:2c > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.10.10.1 tell 10.10.10.2, length 28
22:31:00.897248 fe:1a:a1:78:1a:2e > 6e:61:4d:af:d6:2c, ethertype ARP (0x0806), length 42: Reply 10.10.10.1 is-at fe:1a:a1:78:1a:2e, length 28
22:31:00.897274 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.48951 > 80.237.x.y.53: 54093+ A? google.com. (28)
22:31:05.902463 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.53921 > 80.237.x.y.53: 54093+ A? google.com. (28)
22:31:10.908382 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.48951 > 80.237.x.z.53: 54093+ A? google.com. (28)
22:31:18.270979 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.36427 > 80.237.x.z.53: 52448+ A? google.com. (28)
22:32:30.101175 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.39268 > 80.237.x.z.53: 22787+ A? google.com. (28)
Click to expand...


Hi t_b,

I have no docker instance. i have dedicated server system. i have the problem in the containers. i create a container with "only" an internal ip-adress, and use the proxmox host as gateway. if i enable the proxmox firewall on the containers network card (see attached screenshot), i have no internet connection and no connection to the proxmox host. i think that can not be right. please help!

best regards,

Oliver
 

Attachments

  • container1.png
    container1.png
    25.7 KB · Views: 71
Anyone found a solution for this? I just installed docker on the host and lost internet connection on the Proxmox LXC containers.
 
  • Like
Reactions: noko
eth said:
Anyone found a solution for this? I just installed docker on the host and lost internet connection on the Proxmox LXC containers.
Click to expand...
Hi, I’m in the same situation. But sorry I have no solution yet. I will try soon to install docker in a LXC (w/ nesting, …) hoping it resolves my situation.
 
Docker is somehow conflicting with proxmox routing. Upon scanning the blogs/forums, there is no solution where proxmox and docker can live together on the same level.

I ended up creating a CT (not LXC) with Centos 8 and putting docker inside of it. This is a clean solution, since each level (proxmox/centos/docker) is doing it's job without interfering with each other.

I also needed to passthrough a USB device into docker, so I did a device passthrough with the needed usb bus to the CT via iommu (explained here). No issues whatsoever. This is the reason why I chose the CT route, since it's not possible to cleanly passthrough a device to LXC.
 
Last edited:
  • Like
Reactions: noko
Docker runs fine inside a unprivileged LXC if you enable the nesting and keyctl features (atleast on PVE6.4). But the Proxmox staff recommends to install Docker inside a VM instead.
 
  • Like
Reactions: noko
Can anyone actually explain why does LXC and VM networking breaks when docker (installed on Proxmox host) creates its networks (experienced issues with docker swarm)? Thanks
 
Last edited:
  • Like
Reactions: noko
Code: 
sudo systemctl edit docker

add this:

Code: 
[Service]
ExecStartPost=iptables -I DOCKER-USER -j ACCEPT

save

reboot

probably a huge security risk or wahatever, and not recommended see above. but it works for me
 
You must log in or register to reply here.
Top Bottom