LXC can't access internet with network firewall enabled

[Simon Tong]

Hi,

I've been trying to solve this issue for a few weeks now. I have scoured the internet and come up with no solution.

I am on a dedicated server on the OVH network.

My problem is with the LXC container. I can access it via SSH from my home (via port 2222 as you can see in the prerouting).

But when I am inside the container I cannot access anything external (ping/curl/apt update etc.).

I can ping within the subnet that it is connected to but thats about it.

The only way I can get public access inside the container is to disable the network firewall which is undesired.

My firewall is set up as so:

Datacenter Firewall:

Enabled: yes
INPUT policy: ACCEPT
OUTPUT policy: ACCEPT
Rules (in order):
Proxmox security group (all ports from here allowed pve-proxmox.com/wiki/Ports)
DROP in

Node Firewall:

Enabled: yes
INPUT policy: ACCEPT
OUTPUT policy: ACCEPT
Rules (in order):
Proxmox security group (all ports from here allowed pve-proxmox.com/wiki/Ports)
DROP in

LXC Firewall:

enabled: yes
INPUT policy: DROP
OUTPUT policy: ACCEPT
Rules (in order):
ACCEPT in https/https
ACCEPT in SSH
ACCEPT in Ping

LXC Network:

ID: net0
name: eth0
bridge: vmbr1
Firewall: yes
MAC address: xx:xx:xx:xx:xx:xx
IP: 10.10.10.3
Gateway: 10.10.10.1

----
/etc/network/interfaces set up as so:

Public IP (real ip masked) - 123.123.123.111
Private subnet - 10.10.10.0/24

auto lo
iface lo inet loopback

auto vmbr1
iface vmbr1 inet static
    address  10.10.10.1
    netmask  255.255.255.0
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up   iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp -d 123.123.123.111 --dport 2222 -j DNAT --to 10.10.10.3:22
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp -d 123.123.123.111 --dport 2222 -j DNAT --to 10.10.10.3:22

auto vmbr0
iface vmbr0 inet static
    address  123.123.123.111
    netmask  255.255.255.0
    network 123.123.123.0
    gateway  123.123.123.254
    broadcast  123.123.123.255
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0

I have run tcpdump from within the LXC container. Here is the output:

Note: lxc container hostname is 'websites'

Attempting ping inside container... no response:

$ ping 8.8.8.8

PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
--- 8.8.8.8 ping statistics ---
9 packets transmitted, 0 received, 100% packet loss, time 8110ms

------
TCP output inside container:

$ tcpdump -vv -i eth0 not port 22

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
18:55:58.693991 IP (tos 0x0, ttl 64, id 51340, offset 0, flags [DF], proto ICMP (1), length 84)
    websites.ip-123-123-123.eu > 8.8.8.8: ICMP echo request, id 931, seq 10, length 64
18:55:58.694324 IP (tos 0x0, ttl 64, id 35641, offset 0, flags [DF], proto UDP (17), length 66)
    websites.ip-123-123-123.eu.51231 > 213.186.33.99.domain: [bad udp cksum 0x0b6a -> 0x37fc!] 56041+ PTR? 8.8.8.8.in-addr.arpa. (38)
18:55:59.718020 IP (tos 0x0, ttl 64, id 51549, offset 0, flags [DF], proto ICMP (1), length 84)
    websites.ip-123-123-123.eu > 8.8.8.8: ICMP echo request, id 931, seq 11, length 64
18:56:00.742000 IP (tos 0x0, ttl 64, id 51652, offset 0, flags [DF], proto ICMP (1), length 84)
    websites.ip-123-123-123.eu > 8.8.8.8: ICMP echo request, id 931, seq 12, length 64
18:56:01.766025 IP (tos 0x0, ttl 64, id 51704, offset 0, flags [DF], proto ICMP (1), length 84)
    websites.ip-123-123-123.eu > 8.8.8.8: ICMP echo request, id 931, seq 13, length 64
18:56:02.790017 IP (tos 0x0, ttl 64, id 51726, offset 0, flags [DF], proto ICMP (1), length 84)
    websites.ip-123-123-123.eu > 8.8.8.8: ICMP echo request, id 931, seq 14, length 64
18:56:03.699492 IP (tos 0x0, ttl 64, id 36546, offset 0, flags [DF], proto UDP (17), length 66)
    websites.ip-123-123-123.eu.33827 > 213.186.33.99.domain: [bad udp cksum 0x0b6a -> 0x7bf8!] 56041+ PTR? 8.8.8.8.in-addr.arpa. (38)

Here is the tcpdump from the host:

$ tcpdump -vv -i vmbr0 host websites

tcpdump: listening on vmbr0, link-type EN10MB (Ethernet), capture size 262144 bytes
19:02:58.758025 IP (tos 0x0, ttl 63, id 22511, offset 0, flags [DF], proto ICMP (1), length 84)
    websites > google-public-dns-a.google.com: ICMP echo request, id 954, seq 65, length 64
19:02:59.782030 IP (tos 0x0, ttl 63, id 22736, offset 0, flags [DF], proto ICMP (1), length 84)
    websites > google-public-dns-a.google.com: ICMP echo request, id 954, seq 66, length 64
19:03:00.806040 IP (tos 0x0, ttl 63, id 22933, offset 0, flags [DF], proto ICMP (1), length 84)
    websites > google-public-dns-a.google.com: ICMP echo request, id 954, seq 67, length 64
19:03:01.830032 IP (tos 0x0, ttl 63, id 22982, offset 0, flags [DF], proto ICMP (1), length 84)
    websites > google-public-dns-a.google.com: ICMP echo request, id 954, seq 68, length 64
19:03:02.854036 IP (tos 0x0, ttl 63, id 23091, offset 0, flags [DF], proto ICMP (1), length 84)
    websites > google-public-dns-a.google.com: ICMP echo request, id 954, seq 69, length 64
19:03:03.878028 IP (tos 0x0, ttl 63, id 23187, offset 0, flags [DF], proto ICMP (1), length 84)
    websites > google-public-dns-a.google.com: ICMP echo request, id 954, seq 70, length 64

Any suggestions for how to have outbound traffic work inside the container whilst keeping the network firewall enabled?


Thanks

 

[Simon Tong]

OK,

So after I posted this I found this post which solved my problem:

https://forum.proxmox.com/threads/p...l-with-nat-port-forwarding-hairpinning.24890/

# Allow NAT working with the built-in firewall
post-up   iptables -t raw -I PREROUTING  -i fwbr+ -j CT --zone 1
post-down iptables -t raw -D PREROUTING  -i fwbr+ -j CT --zone 1