NAT and firewall for LXC CTs

[jeannotp]

Hello,

I searched but didn't find the response to this question:

Can built-in firewall be used with LXC CTs behind NAT?

I run a fresh PVE/4.1-1/2f9650d4 (running kernel: 4.2.6-1-pve) and have the following config:

# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    address 198.51.11.20
    netmask 255.255.255.0
    gateway 198.51.11.1

auto vmbr0
iface vmbr0 inet static
    address 10.10.8.1
    netmask 255.255.248.0
    bridge_ports none
    bridge_stp off
    bridge_fd 0

        post-up /bin/echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   /sbin/iptables -t nat -A POSTROUTING -s '10.10.8.0/21' -o eth0 -j MASQUERADE
        post-down /sbin/iptables -t nat -D POSTROUTING -s '10.10.8.0/21' -o eth0 -j MASQUERADE

Datacenter level:

• Enable Firewall: Yes
• Input Policy: DROP
• Output Policy: ACCEPT

Node level: (I have only one node)

• Enable Firewall: Yes
• all defaults settings

Firewall works correctly.
But for my LXC containers, when I enable the firewall for the CT, I cannot anymore ping WAN hosts.

With tcpdump, I see that when the FW is enabled, ICMP requests from the CT are not translated and are sent on the WAN interface.

PVE Node # tcpdump host 10.10.10.104
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
22:32:59.612224 IP 10.10.10.104 > bidule-public-dns-a.bidule.com: ICMP echo request, id 42498, seq 15, length 64
22:33:00.620198 IP 10.10.10.104 > bidule-public-dns-a.bidule.com: ICMP echo request, id 42498, seq 16, length 64
22:33:01.628221 IP 10.10.10.104 > bidule-public-dns-a.bidule.com: ICMP echo request, id 42498, seq 17, length 64

There's maybe a really simple trick to do but I don't find it.

Thanks you!

P.S. : I think I've found.

Cf. https://forum.proxmox.com/threads/p...l-with-nat-port-forwarding-hairpinning.24890/

# Allow NAT working with the built-in firewall
        post-up   iptables -t raw -I PREROUTING  -i fwbr+ -j CT --zone 1
        post-down iptables -t raw -D PREROUTING  -i fwbr+ -j CT --zone 1

I'll check and mark the thread as solved if it's OK.

P.P.S. : It seems OK, I mark the tread as solved.

 

[jeannotp]

I reopen this thread because I see some things I don't understand.
At container's boot, maybe not all the time but often, the CT has no Internet access (I ping a reliable IP).

10.10.8.1 is vmbr0 gateway (4e:dd:39:00:39:2f)
wiki is CT's name

(tcpdump arp running on CT)

20:53:34.738665 ARP, Request who-has 10.10.8.1 tell wiki, length 28
20:53:34.738707 ARP, Reply 10.10.8.1 is-at de:3b:29:46:d3:85 (oui Unknown), length 28
20:53:39.750679 ARP, Request who-has 10.10.10.105 tell wiki, length 28
20:53:39.750719 ARP, Request who-has wiki tell 10.10.10.105, length 28
20:53:39.750735 ARP, Reply wiki is-at 36:65:65:65:65:38 (oui Unknown), length 28
20:53:39.750764 ARP, Reply 10.10.10.105 is-at 66:38:63:66:63:35 (oui Unknown), length 28
20:54:21.858781 ARP, Request who-has 10.10.8.1 tell wiki, length 28
20:54:21.858815 ARP, Reply 10.10.8.1 is-at de:3b:29:46:d3:85 (oui Unknown), length 28
I disable firewall for the CT.
20:55:09.218630 ARP, Request who-has 10.10.8.1 tell wiki, length 28
20:55:10.218706 ARP, Request who-has 10.10.8.1 tell wiki, length 28
20:55:11.218739 ARP, Request who-has 10.10.8.1 tell wiki, length 28
20:55:12.412950 ARP, Request who-has 10.10.8.1 tell wiki, length 28
20:55:12.412998 ARP, Reply 10.10.8.1 is-at 4e:dd:39:00:39:2f (oui Unknown), length 28
20:55:12.413103 ARP, Reply 10.10.8.1 is-at 1a:e5:13:62:aa:30 (oui Unknown), length 28
20:55:12.413107 ARP, Reply 10.10.8.1 is-at 96:9b:89:2e:b5:1b (oui Unknown), length 28
20:55:12.413112 ARP, Reply 10.10.8.1 is-at 1a:31:fb:86:b6:13 (oui Unknown), length 28
20:55:12.413116 ARP, Reply 10.10.8.1 is-at 8a:b4:b5:04:f4:47 (oui Unknown), length 28
20:55:12.413121 ARP, Reply 10.10.8.1 is-at 46:96:d5:75:33:2d (oui Unknown), length 28
20:55:12.413129 ARP, Reply 10.10.8.1 is-at de:8d:bb:42:d0:97 (oui Unknown), length 28
20:55:12.413133 ARP, Reply 10.10.8.1 is-at a2:80:5c:83:1e:a8 (oui Unknown), length 28
20:55:12.413138 ARP, Reply 10.10.8.1 is-at 76:52:49:87:31:ea (oui Unknown), length 28
20:55:12.413142 ARP, Reply 10.10.8.1 is-at c2:c9:84:f8:cd:18 (oui Unknown), length 28
20:55:12.413147 ARP, Reply 10.10.8.1 is-at f2:4a:d0:2f:c5:e8 (oui Unknown), length 28
20:55:12.413153 ARP, Reply 10.10.8.1 is-at 6e:80:2a:0e:6c:bc (oui Unknown), length 28
20:55:21.522596 ARP, Request who-has wiki tell 10.10.8.1, length 28
20:55:21.522625 ARP, Reply wiki is-at 36:65:65:65:65:38 (oui Unknown), length 28

And after a while I can ping the outside.
And then I can enable the firewall. But it doesn't last.

It musn't be very unusual to use NAT with firewall and CTs.
Am I the only one to see this?
My /etc/network/interfaces config and my iptables rules are maybe bad ones?

P.S. : One turn-around would be to create a static ARP mapping in each CT.

ip neighbor add 10.10.8.1 lladdr 4e:dd:39:00:39:2f dev eth0 nud permanent # if non existent
ip neighbor change 10.10.8.1 lladdr 4e:dd:39:00:39:2f dev eth0 # if existent

Very elegant..