Firewall enabled interface can't ping outside (routing?)

[Bkelzo]

Hi,

I have a single Proxmox 5.3-12 host with a couple of bridges (one for each network), e.g.

eth0: public host IP / gateway
vmbr0: 10.10.10.0/24
...
vmbr4: 10.10.50.0/24

and so on.

I'm trying to configure the firewall settings but as soon as I enable the firewall option in the VMs network interface, it cannot ping any outside servers such as 8.8.8.8.

I have currently two Ubuntu 18.04 VMs for comparison, both on vmbr4, one with firewall enabled and with without it. For both VMs, all other firewall options are either set to "no", or "ACCEPT". The same goes for cluster and host, so as far as I understand, no firewall rules should be active.

I also flushed all IPtables rules and added just the following bare necessities.

iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
ACCEPT     tcp  --  XX.XXX.XX.XX        0.0.0.0/0            tcp dpt:22 /* Allow SSH from XXXX */
ACCEPT     all  --  127.0.0.1            0.0.0.0/0            /* Allow all connections from localhost */
ACCEPT     all  --  10.10.0.0/16         0.0.0.0/0            /* Allow all connections from local nets */
DROP       all  --  0.0.0.0/0            0.0.0.0/0            /* Drop all other incoming external requests */

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

(XXX being redacted)

For NAT, I've only added the following to allow VMs to connect to the outside world.

Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  all  --  10.10.0.0/16         0.0.0.0/0

For any VM, this works just as intended. The only exceptions are VMs that have the firewall flag enabled and I cannot figure out why this doesn't work. They can ping the internal Host interface (i.e. 10.10.50.1) but any ping to 8.8.8.8 doesn't receive a response. I'm not using any VLANs, so that field is always empty.

The PVE host shows this when I run TCPDUMP on the vmbr4 interface.

Host 10.10.50.101 without firewall

# tcpdump -i vmbr4 icmp -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr4, link-type EN10MB (Ethernet), capture size 262144 bytes
19:34:58.339009 IP 10.10.50.101 > 8.8.8.8: ICMP echo request, id 3443, seq 1, length 64
19:34:58.344180 IP 8.8.8.8 > 10.10.50.101: ICMP echo reply, id 3443, seq 1, length 64
19:34:59.340455 IP 10.10.50.101 > 8.8.8.8: ICMP echo request, id 3443, seq 2, length 64
19:34:59.345583 IP 8.8.8.8 > 10.10.50.101: ICMP echo reply, id 3443, seq 2, length 64
19:35:00.341688 IP 10.10.50.101 > 8.8.8.8: ICMP echo request, id 3443, seq 3, length 64
19:35:00.346827 IP 8.8.8.8 > 10.10.50.101: ICMP echo reply, id 3443, seq 3, length 64
19:35:01.343274 IP 10.10.50.101 > 8.8.8.8: ICMP echo request, id 3443, seq 4, length 64
19:35:01.348432 IP 8.8.8.8 > 10.10.50.101: ICMP echo reply, id 3443, seq 4, length 64

and against 10.10.50.100 with firewall enabled for the network interface (but nowhere else)

# tcpdump -i vmbr4 icmp -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr4, link-type EN10MB (Ethernet), capture size 262144 bytes

19:35:08.017394 IP 10.10.50.100 > 8.8.8.8: ICMP echo request, id 1424, seq 1, length 64
19:35:09.022814 IP 10.10.50.100 > 8.8.8.8: ICMP echo request, id 1424, seq 2, length 64
19:35:10.046865 IP 10.10.50.100 > 8.8.8.8: ICMP echo request, id 1424, seq 3, length 64
19:35:11.070818 IP 10.10.50.100 > 8.8.8.8: ICMP echo request, id 1424, seq 4, length 64

In case it helps, this is what ip addr shows me for these VMs. VM 501 with tap501i0 has no problems, but VM 500 does as detailed above.

68: tap501i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr4 state UNKNOWN group default qlen 1000
    link/ether fe:b3:af:ee:83:50 brd ff:ff:ff:ff:ff:ff
74: tap500i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr500i0 state UNKNOWN group default qlen 1000
    link/ether 8e:77:6f:0f:49:b3 brd ff:ff:ff:ff:ff:ff
75: fwbr500i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 02:b8:77:cd:df:a4 brd ff:ff:ff:ff:ff:ff
76: fwpr500p0@fwln500i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr4 state UP group default qlen 1000
    link/ether 62:c3:88:16:e9:ef brd ff:ff:ff:ff:ff:ff
77: fwln500i0@fwpr500p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr500i0 state UP group default qlen 1000
    link/ether 02:b8:77:cd:df:a4 brd ff:ff:ff:ff:ff:ff

I hope someone here can point me in the right direction because this is driving me nuts... thanks.

 

[mira]

Please post the output of 'iptables-save' and your /etc/network/interfaces.

 

[spirit]

With nat/masquerade + firewall , you need to setup conntrack zones ( -j CT ) in prerouting and posrouting. Search in the forum.

 

[Bkelzo]

Thanks for the replies. After tinkering with it for quite some time, I decided to drop it and have switched from iptables to nftables. Everything is working fine and is neatly stored in one config. Downside is, I can't use the proxmox interface but at least my firewall is doing what I want from it now.

@spirit, thanks for the hint with conntrack zones. If I ever revisit this topic, I'll have a look at that.