No internet connection in container via NAT

L

Luki20

Well-Known Member
Apr 28, 2018
44
1
48
38
As many other users have asked before, I also have a connection problem in my container via NAT. I searched all ~30 threads but could not find an solution.

What does work:
- pinging from 10.0.0.2 (container) to host (10.0.0.1)
- pinging from host (10.0.0.1) to 10.0.0.2 (container)
- pinging from 10.0.0.2 (container1) to 10.0.0.3 (container2) and back

What does not work:
- pinging from container1 or container2 to public IPs like 8.8.8.8 or 1.1.1.1 (google/cloudflare, for example)

My proxmox system:
Code: 
Kernel Version: Linux 4.15.18-21-pve #1 SMP PVE 4.15.18-48 (Fri, 20 Sep 2019 11:28:30 +0200)
PVE Manager Version: pve-manager/5.4-13/aee6f0ec

Let's start with my config at first (from /etc/network/interfaces):
Code: 
source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback
iface lo inet6 loopback
iface enp2s0 inet manual
        up route add -net xxx.xxx.xxx.192 netmask 255.255.255.192 gw xxx.xxx.xxx.193 dev enp2s0
# route xxx.xxx.xxx.192/26 via xxx.xxx.xxx.193

auto vmbr0
iface vmbr0 inet static
        address  xxx.xxx.xxx.227
        netmask  255.255.255.192
        gateway  xxx.xxx.xxx.193
        bridge-ports enp2s0
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
    address 10.0.0.1
    netmask 255.255.255.0
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE

Container is configured as following:
Code: 
net0: name=eth0,bridge=vmbr1,gw=10.0.0.1,hwaddr=EE:63:4E:C8:84:75,ip=10.0.0.2/24,type=veth

IPv4 forward is enabled:
Code: 
~ sysctl -p /etc/sysctl.conf

net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0

tcpdump while pinging to 1.1.1.1 shows this:
Code: 
~ tcpdump -i vmbr1

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr1, link-type EN10MB (Ethernet), capture size 262144 bytes
23:53:18.371528 IP 10.0.0.2 > 10.0.0.1: ICMP echo request, id 368, seq 1, length 64
23:53:18.371570 IP 10.0.0.1 > 10.0.0.2: ICMP echo reply, id 368, seq 1, length 64
23:53:19.402145 IP 10.0.0.2 > 10.0.0.1: ICMP echo request, id 368, seq 2, length 64
23:53:19.402177 IP 10.0.0.1 > 10.0.0.2: ICMP echo reply, id 368, seq 2, length 64
23:53:23.434095 ARP, Request who-has 10.0.0.2 tell 10.0.0.1, length 28
23:53:23.434124 ARP, Request who-has 10.0.0.1 tell 10.0.0.2, length 28
23:53:23.434200 ARP, Reply 10.0.0.1 is-at fe:bf:2f:35:cf:57 (oui Unknown), length 28
23:53:23.434193 ARP, Reply 10.0.0.2 is-at ee:63:4e:c8:84:75 (oui Unknown), length 28
23:53:32.981362 IP 10.0.0.2.52622 > dns.google.domain: 5037+ A? 1.1.1. (23)
23:53:32.981391 IP 10.0.0.2.52622 > dns.google.domain: 13749+ AAAA? 1.1.1. (23)
23:53:36.042106 IP 10.0.0.1 > 10.0.0.2: ICMP host dns.google unreachable, length 59
23:53:36.042122 IP 10.0.0.1 > 10.0.0.2: ICMP host dns.google unreachable, length 59
23:53:37.244937 IP 10.0.0.2 > one.one.one.one: ICMP echo request, id 370, seq 1, length 64
23:53:38.250173 IP 10.0.0.2 > one.one.one.one: ICMP echo request, id 370, seq 2, length 64
23:53:39.274177 IP 10.0.0.2 > one.one.one.one: ICMP echo request, id 370, seq 3, length 64
23:53:40.298105 IP 10.0.0.1 > 10.0.0.2: ICMP host one.one.one.one unreachable, length 92
23:53:40.298121 IP 10.0.0.1 > 10.0.0.2: ICMP host one.one.one.one unreachable, length 92
23:53:40.298126 IP 10.0.0.1 > 10.0.0.2: ICMP host one.one.one.one unreachable, length 92
23:53:40.298176 IP 10.0.0.2 > one.one.one.one: ICMP echo request, id 370, seq 4, length 64
23:53:41.322130 IP 10.0.0.2 > one.one.one.one: ICMP echo request, id 370, seq 5, length 64
23:53:43.370100 IP 10.0.0.1 > 10.0.0.2: ICMP host one.one.one.one unreachable, length 92
23:53:43.370111 IP 10.0.0.1 > 10.0.0.2: ICMP host one.one.one.one unreachable, length 92
23:53:45.450085 ARP, Request who-has 10.0.0.2 tell 10.0.0.1, length 28
23:53:45.450114 ARP, Reply 10.0.0.2 is-at ee:63:4e:c8:84:75 (oui Unknown), length 28
^C
24 packets captured
24 packets received by filter
0 packets dropped by kernel

I don't know what to do now. I really troubleshooted everything.

Has somebody a idea what to try next here? Thanks!
 
Last edited:
maxir200 said:
As many other users have asked before, I also have a connection problem in my container via NAT. I searched all ~30 threads but could not find an solution.

What does work:
- pinging from 10.0.0.2 (container) to host (10.0.0.1)
- pinging from host (10.0.0.1) to 10.0.0.2 (container)
- pinging from 10.0.0.2 (container1) to 10.0.0.3 (container2) and back

What does not work:
- pinging from container1 or container2 to public IPs like 8.8.8.8 or 1.1.1.1 (google/cloudflare, for example)

My proxmox system:
Code: 
Kernel Version: Linux 4.15.18-21-pve #1 SMP PVE 4.15.18-48 (Fri, 20 Sep 2019 11:28:30 +0200)
PVE Manager Version: pve-manager/5.4-13/aee6f0ec

Let's start with my config at first (from /etc/network/interfaces):
Code: 
source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback
iface lo inet6 loopback
iface enp2s0 inet manual
        up route add -net xxx.xxx.xxx.192 netmask 255.255.255.192 gw xxx.xxx.xxx.193 dev enp2s0
# route xxx.xxx.xxx.192/26 via xxx.xxx.xxx.193

auto vmbr0
iface vmbr0 inet static
        address  xxx.xxx.xxx.227
        netmask  255.255.255.192
        gateway  xxx.xxx.xxx.193
        bridge-ports enp2s0
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
    address 10.0.0.1
    netmask 255.255.255.0
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE

Container is configured as following:
Code: 
net0: name=eth0,bridge=vmbr1,gw=10.0.0.1,hwaddr=EE:63:4E:C8:84:75,ip=10.0.0.2/24,type=veth

IPv4 forward is enabled:
Code: 
~ sysctl -p /etc/sysctl.conf

net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0

tcpdump while pinging to 1.1.1.1 shows this:
Code: 
~ tcpdump -i vmbr1

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr1, link-type EN10MB (Ethernet), capture size 262144 bytes
23:53:18.371528 IP 10.0.0.2 > 10.0.0.1: ICMP echo request, id 368, seq 1, length 64
23:53:18.371570 IP 10.0.0.1 > 10.0.0.2: ICMP echo reply, id 368, seq 1, length 64
23:53:19.402145 IP 10.0.0.2 > 10.0.0.1: ICMP echo request, id 368, seq 2, length 64
23:53:19.402177 IP 10.0.0.1 > 10.0.0.2: ICMP echo reply, id 368, seq 2, length 64
23:53:23.434095 ARP, Request who-has 10.0.0.2 tell 10.0.0.1, length 28
23:53:23.434124 ARP, Request who-has 10.0.0.1 tell 10.0.0.2, length 28
23:53:23.434200 ARP, Reply 10.0.0.1 is-at fe:bf:2f:35:cf:57 (oui Unknown), length 28
23:53:23.434193 ARP, Reply 10.0.0.2 is-at ee:63:4e:c8:84:75 (oui Unknown), length 28
23:53:32.981362 IP 10.0.0.2.52622 > dns.google.domain: 5037+ A? 1.1.1. (23)
23:53:32.981391 IP 10.0.0.2.52622 > dns.google.domain: 13749+ AAAA? 1.1.1. (23)
23:53:36.042106 IP 10.0.0.1 > 10.0.0.2: ICMP host dns.google unreachable, length 59
23:53:36.042122 IP 10.0.0.1 > 10.0.0.2: ICMP host dns.google unreachable, length 59
23:53:37.244937 IP 10.0.0.2 > one.one.one.one: ICMP echo request, id 370, seq 1, length 64
23:53:38.250173 IP 10.0.0.2 > one.one.one.one: ICMP echo request, id 370, seq 2, length 64
23:53:39.274177 IP 10.0.0.2 > one.one.one.one: ICMP echo request, id 370, seq 3, length 64
23:53:40.298105 IP 10.0.0.1 > 10.0.0.2: ICMP host one.one.one.one unreachable, length 92
23:53:40.298121 IP 10.0.0.1 > 10.0.0.2: ICMP host one.one.one.one unreachable, length 92
23:53:40.298126 IP 10.0.0.1 > 10.0.0.2: ICMP host one.one.one.one unreachable, length 92
23:53:40.298176 IP 10.0.0.2 > one.one.one.one: ICMP echo request, id 370, seq 4, length 64
23:53:41.322130 IP 10.0.0.2 > one.one.one.one: ICMP echo request, id 370, seq 5, length 64
23:53:43.370100 IP 10.0.0.1 > 10.0.0.2: ICMP host one.one.one.one unreachable, length 92
23:53:43.370111 IP 10.0.0.1 > 10.0.0.2: ICMP host one.one.one.one unreachable, length 92
23:53:45.450085 ARP, Request who-has 10.0.0.2 tell 10.0.0.1, length 28
23:53:45.450114 ARP, Reply 10.0.0.2 is-at ee:63:4e:c8:84:75 (oui Unknown), length 28
^C
24 packets captured
24 packets received by filter
0 packets dropped by kernel

I don't know what to do now. I really troubleshooted everything.

Has somebody a idea what to try next here? Thanks!
Click to expand...


Looks like the default route is missed. Check what
Code: 
ip route

reports as well as having a look what happens at vmbr0 via

Code: 
tcpdump -e -n -i vmbr0


Btw.: recommend to use -n option for tcpdump (name resolution is rather confusing than helping in this case).


Firewall active?
 
Thanks for your first investigation.

Richard said:
Looks like the default route is missed. Check what
Click to expand...
Code: 
~ # ip route

default via 148.xxx.xxx.193 dev enp2s0 proto static
default via 148.xxx.xxx.193 dev vmbr0 proto static
10.0.0.0/24 dev vmbr1 proto kernel scope link src 10.0.0.1
148.xxx.xxx.192/26 dev vmbr0 proto kernel scope link src 148.xxx.xxx.227
148.xxx.xxx.193 dev enp2s0 proto kernel scope link src 148.xxx.xxx.227
148.xxx.xxx.193 dev vmbr0 proto kernel scope link src 148.xxx.xxx.227

Richard said:
reports as well as having a look what happens at vmbr0 via
Click to expand...

There is so much happening there - which things should I post/filter to help you further?

Richard said:
Firewall active?
Click to expand...

Disabled at datacenter-, node- and LXC-level. But at the datacenter-firewall are some filters (SMURFS-filter) enabled. Do they have some impact or are they really enabled when firewall is off?
 
maxir200 said:
Code: 
~ # ip route

default via 148.xxx.xxx.193 dev enp2s0 proto static
default via 148.xxx.xxx.193 dev vmbr0 proto static
10.0.0.0/24 dev vmbr1 proto kernel scope link src 10.0.0.1
148.xxx.xxx.192/26 dev vmbr0 proto kernel scope link src 148.xxx.xxx.227
148.xxx.xxx.193 dev enp2s0 proto kernel scope link src 148.xxx.xxx.227
148.xxx.xxx.193 dev vmbr0 proto kernel scope link src 148.xxx.xxx.227
Click to expand...


The (default) routes should be assigned to vmbr0 only, not to enp2s0.

maxir200 said:
There is so much happening there - which things should I post/filter to help you further?
Click to expand...

Filter e.g. by source resp. destination address:
Code: 
tcpdump host 8.8.8.8 -e -n -i vmbr0



maxir200 said:
Disabled at datacenter-, node- and LXC-level. But at the datacenter-firewall are some filters (SMURFS-filter) enabled. Do they have some impact or are they really enabled when firewall is off?
Click to expand...


If "datacenter" firewall is disabled there cannot be any impacts, if it would be enabled they can have.
 
Richard said:
The (default) routes should be assigned to vmbr0 only, not to enp2s0.
Click to expand...

So I only have to remove the first entry with ip route del? Is this change permanent or lost after reboot?


Richard said:
tcpdump host 8.8.8.8 -e -n -i vmbr0
Click to expand...

When pinging to 8.8.8.8 in the container while a running tcpdump nothing is shown there.

Richard said:
If "datacenter" firewall is disabled there cannot be any impacts, if it would be enabled they can have.
Click to expand...

Okay, then everything should be okay there.
 
When pinging to 8.8.8.8 in the container while a running tcpdump nothing is shown there.
Click to expand...
This is a problem. Please take a look at this setup:

Code: 
➜  ~ cat /etc/network/interfaces
auto lo
iface lo inet loopback

iface ens18 inet manual

auto vmbr0
iface vmbr0 inet static
    address  192.168.25.130
    netmask  255.255.240.0
    gateway  192.168.16.1
    bridge-ports ens18
    bridge-stp off
    bridge-fd 0

auto vmbr1
iface vmbr1 inet static
    address  10.0.0.1
    netmask  24
    bridge-ports none
    bridge-stp off
    bridge-fd 0

➜  ~ pct config 100 | grep net
net0: name=eth0,bridge=vmbr1,firewall=1,gw=10.0.0.1,hwaddr=4A:AD:F7:46:81:7D,ip=10.0.0.2/24,type=veth
➜  ~ pct config 101 | grep net
net0: name=eth0,bridge=vmbr1,firewall=1,gw=10.0.0.1,hwaddr=42:62:D3:D3:73:07,ip=10.0.0.3/24,type=veth
➜  ~ ip route
default via 192.168.16.1 dev vmbr0 onlink 
10.0.0.0/24 dev vmbr1 proto kernel scope link src 10.0.0.1 
192.168.16.0/20 dev vmbr0 proto kernel scope link src 192.168.25.130

If you ping 8.8.8.8 from the container then
Code: 
tcpdump -e -n -i vmbr0 host 8.8.8.8
does not show anything at first. However, if you execute
Code: 
echo 1 > /proc/sys/net/ipv4/ip_forward
on the host then tcpdump should show something.
 
Thanks for your reply, but nothing helped.

I wanted to give an update that I reinstalled Proxmox last week and now everything works fine. I really don't know what the problem was here.
 
  • Like
Reactions: Dominic
You must log in or register to reply here.
Top Bottom