no connection to the internet from container

T

t_b

New Member
Nov 4, 2015
22
2
1
Hello,

I currenty facing network problems with the virtual adapter for the container network.
Here is my network config

/etc/network/interfaces
Code: 
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address 5.35.x.y
        netmask 255.255.255.0
        gateway 5.35.x.y

auto vmbr0
iface vmbr0 inet static
        address  10.0.0.1
        netmask  255.255.255.0
        bridge_ports none
        bridge_stp off
        bridge_fd 0

/etc/sysctl.conf
Code: 
net.ipv4.ip_forward=1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0

ip addr (proxmox)
Code: 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:1c:42:74:61:bd brd ff:ff:ff:ff:ff:ff
    inet 5.35.x.y/24 brd 5.35.x.y scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::21c:x:x:x/64 scope link 
       valid_lft forever preferred_lft forever
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fe:30:d6:f5:bd:91 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.1/24 brd 10.0.0.255 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::e44e:x:x:x/64 scope link 
       valid_lft forever preferred_lft forever
6: veth100i0@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether fe:30:d6:f5:bd:91 brd ff:ff:ff:ff:ff:ff link-netnsid 0

ping (proxmox)
Code: 
root@proxmox:~# ping 10.0.0.2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_seq=1 ttl=64 time=0.094 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=64 time=0.068 ms
^C
--- 10.0.0.2 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.068/0.081/0.094/0.013 ms
root@proxmox:~# ping google.com
PING google.com (172.217.23.174) 56(84) bytes of data.
64 bytes from fra15s22-in-f174.1e100.net (172.217.23.174): icmp_seq=1 ttl=56 time=3.84 ms
64 bytes from fra15s22-in-f174.1e100.net (172.217.23.174): icmp_seq=2 ttl=56 time=4.02 ms


ip addr (container)
Code: 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
7: eth0@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 6e:61:4d:af:d6:2c brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.2/24 brd 10.0.0.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::6c61:4dff:feaf:d62c/64 scope link 
       valid_lft forever preferred_lft forever

ping (container)
Code: 
root@container:~# ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.196 ms
64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.072 ms
^C
--- 10.0.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.072/0.134/0.196/0.062 ms
root@container:~# ping google.com
^C

I Thought it's a firewall problem so i disabled firewall on cluster & node but it made no difference. But in case here is the firewall config:

/etc/pve/firewall/cluster.fw
Code: 
[OPTIONS]

policy_in: ACCEPT
enable: 1

[RULES]

GROUP proxmox
IN ACCEPT -p tcp -dport 22 # ssh
IN DROP

[group proxmox]

IN ACCEPT -p tcp -dport 22 # ssh
IN ACCEPT -p tcp -dport 443 # https
IN ACCEPT -p tcp -dport 80 # http
 
your config looks OK to me
simple question, did you setup a DNS resolver in the container ?
ie, what is the output of /etc/resolv.conf ?
 
I've used the dns tab on the ui which generated following content in the container
Code: 
# --- BEGIN PVE ---
domain.tld
nameserver 80.237.x.y
nameserver 80.237.x.z
# --- END PVE ---

This are the nameserver from the hosting provider
 
manu said:
It seems you miss the masquerading bits in /etc/network/interfaces

see https://pve.proxmox.com/wiki/Network_Model, and look for "masquerade" in that page
Click to expand...

Hi,

I have a same issue.

I had no internet connection if i enable the firewall on the network card on my container. the container use my proxmox-host as gateway, to connect to the internet. if i enable the firewall on my container i have no connection. i can't ping also the proxmox host. the container have no public ip-adress.

more information you find in the post.

https://forum.proxmox.com/threads/proxmox-firewall-problem-container-bridges.33166/#post-163245


see also my attached screenshot.
Best regards,

Oliver
 

Attachments

  • container1.png
    container1.png
    25.7 KB · Views: 118
@manu
I've already tried that, but i've updated the vmbr0 section again with

Code: 
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eth0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eth0 -j MASQUERADE

I restart the network with
Code: 
systemctl restart networking

also i stopped an started the container... same issue

EDIT:

What i not mentioned is that i'm using docker on the proxmox host. Here is the outut of iptables-save:

Code: 
# Generated by iptables-save v1.4.21 on Mon Mar  6 12:36:25 2017
*filter
:INPUT ACCEPT [197103:61219441]
:FORWARD ACCEPT [1:56]
:OUTPUT ACCEPT [199328:66915477]
:DOCKER - [0:0]
:DOCKER-ISOLATION - [0:0]
-A FORWARD -j DOCKER-ISOLATION
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 9000 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8765 -j ACCEPT
-A DOCKER -d 172.17.0.4/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8080 -j ACCEPT
-A DOCKER-ISOLATION -j RETURN
COMMIT
# Completed on Mon Mar  6 12:36:25 2017
# Generated by iptables-save v1.4.21 on Mon Mar  6 12:36:25 2017
*nat
:PREROUTING ACCEPT [22:1607]
:INPUT ACCEPT [21:1551]
:OUTPUT ACCEPT [252:15240]
:POSTROUTING ACCEPT [252:15240]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.17.0.2/32 -d 172.17.0.2/32 -p tcp -m tcp --dport 9000 -j MASQUERADE
-A POSTROUTING -s 172.17.0.3/32 -d 172.17.0.3/32 -p tcp -m tcp --dport 8765 -j MASQUERADE
-A POSTROUTING -s 172.17.0.4/32 -d 172.17.0.4/32 -p tcp -m tcp --dport 8080 -j MASQUERADE
-A POSTROUTING -s 10.0.0.0/24 -o eth0 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
-A DOCKER -d 127.0.0.1/32 ! -i docker0 -p tcp -m tcp --dport 9000 -j DNAT --to-destination 172.17.0.2:9000
-A DOCKER -d 127.0.0.1/32 ! -i docker0 -p tcp -m tcp --dport 8765 -j DNAT --to-destination 172.17.0.3:8765
-A DOCKER -d 127.0.0.1/32 ! -i docker0 -p tcp -m tcp --dport 8080 -j DNAT --to-destination 172.17.0.4:8080
COMMIT
# Completed on Mon Mar  6 12:36:25 2017
 
@manu You're right, that would be better, but since it's already a "virtualized root server" i'm not abled to create KVM machines, only containers. On the other hand @OliverB seem not having docker and running into the same issues. I'm willing to test remove the docker installation restart the complete environment and re-check the network connection. If it's not working it's probably not a docker issue.
 
@manu So, docker is away until the problems solved - since docker seems not to be the problem. I tried a lot and nothing helped. i compared the configuration with this of my personal instance with no luck.
 
@OliverB BTW, i'm getting exactly the same responses:


Code: 
22:31:00.897158 6e:61:4d:af:d6:2c > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.10.10.1 tell 10.10.10.2, length 28
22:31:00.897248 fe:1a:a1:78:1a:2e > 6e:61:4d:af:d6:2c, ethertype ARP (0x0806), length 42: Reply 10.10.10.1 is-at fe:1a:a1:78:1a:2e, length 28
22:31:00.897274 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.48951 > 80.237.x.y.53: 54093+ A? google.com. (28)
22:31:05.902463 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.53921 > 80.237.x.y.53: 54093+ A? google.com. (28)
22:31:10.908382 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.48951 > 80.237.x.z.53: 54093+ A? google.com. (28)
22:31:18.270979 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.36427 > 80.237.x.z.53: 52448+ A? google.com. (28)
22:32:30.101175 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.39268 > 80.237.x.z.53: 22787+ A? google.com. (28)
 
t_b said:
@OliverB BTW, i'm getting exactly the same responses:


Code: 
22:31:00.897158 6e:61:4d:af:d6:2c > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.10.10.1 tell 10.10.10.2, length 28
22:31:00.897248 fe:1a:a1:78:1a:2e > 6e:61:4d:af:d6:2c, ethertype ARP (0x0806), length 42: Reply 10.10.10.1 is-at fe:1a:a1:78:1a:2e, length 28
22:31:00.897274 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.48951 > 80.237.x.y.53: 54093+ A? google.com. (28)
22:31:05.902463 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.53921 > 80.237.x.y.53: 54093+ A? google.com. (28)
22:31:10.908382 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.48951 > 80.237.x.z.53: 54093+ A? google.com. (28)
22:31:18.270979 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.36427 > 80.237.x.z.53: 52448+ A? google.com. (28)
22:32:30.101175 6e:61:4d:af:d6:2c > fe:1a:a1:78:1a:2e, ethertype IPv4 (0x0800), length 70: 10.10.10.2.39268 > 80.237.x.z.53: 22787+ A? google.com. (28)
Click to expand...


Hi t_b,

I have no docker instance. i have dedicated server system. i have the problem in the containers. i create a container with "only" an internal ip-adress, and use the proxmox host as gateway. if i enable the proxmox firewall on the containers network card (see attached screenshot), i have no internet connection and no connection to the proxmox host. i think that can not be right. please help!

best regards,

Oliver
 

Attachments

  • container1.png
    container1.png
    25.7 KB · Views: 71
Anyone found a solution for this? I just installed docker on the host and lost internet connection on the Proxmox LXC containers.
 
  • Like
Reactions: noko
eth said:
Anyone found a solution for this? I just installed docker on the host and lost internet connection on the Proxmox LXC containers.
Click to expand...
Hi, I’m in the same situation. But sorry I have no solution yet. I will try soon to install docker in a LXC (w/ nesting, …) hoping it resolves my situation.
 
Docker is somehow conflicting with proxmox routing. Upon scanning the blogs/forums, there is no solution where proxmox and docker can live together on the same level.

I ended up creating a CT (not LXC) with Centos 8 and putting docker inside of it. This is a clean solution, since each level (proxmox/centos/docker) is doing it's job without interfering with each other.

I also needed to passthrough a USB device into docker, so I did a device passthrough with the needed usb bus to the CT via iommu (explained here). No issues whatsoever. This is the reason why I chose the CT route, since it's not possible to cleanly passthrough a device to LXC.
 
Last edited:
  • Like
Reactions: noko
Docker runs fine inside a unprivileged LXC if you enable the nesting and keyctl features (atleast on PVE6.4). But the Proxmox staff recommends to install Docker inside a VM instead.
 
  • Like
Reactions: noko
Can anyone actually explain why does LXC and VM networking breaks when docker (installed on Proxmox host) creates its networks (experienced issues with docker swarm)? Thanks
 
Last edited:
  • Like
Reactions: noko
Code: 
sudo systemctl edit docker

add this:

Code: 
[Service]
ExecStartPost=iptables -I DOCKER-USER -j ACCEPT

save

reboot

probably a huge security risk or wahatever, and not recommended see above. but it works for me
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back