Firewall not working on a single NIC out of two?

Progratron · Mar 16, 2019

Since hours trying to figure out what's wrong... I have two NICs both connected to vmbr0 (one with private and one with public IP). Here is how it's configured in proxmox GUI:

Which corresponds to this within the CT-VM:

Code:

auto lo eth0
iface lo inet loopback

iface eth0 inet dhcp

#auto eth1
#iface eth1 inet dhcp

auto eth1
iface eth1 inet static
    address X.X.X.X
    netmask 255.255.255.248

I am aware that I have to enable the firewall through the whole cascade (cluster-node-vm-nic). My problem is that I want to enable it only for net1 and when I do it - it does nothing. And suddenly, when I enable it for net0 as well (just for testing) it starts working, blocking the traffic on both interfaces (which I don't need). What am I doing wrong?

What is more strange - exactly the same setup with Windows VM does the job perfectly.

Richard · Mar 18, 2019

Progratron said:
Since hours trying to figure out what's wrong... I have two NICs both connected to vmbr0 (one with private and one with public IP). Here is how it's configured in proxmox GUI:

Which corresponds to this within the CT-VM:

Code:

auto lo eth0 iface lo inet loopback iface eth0 inet dhcp #auto eth1 #iface eth1 inet dhcp auto eth1 iface eth1 inet static address X.X.X.X netmask 255.255.255.248

I am aware that I have to enable the firewall through the whole cascade (cluster-node-vm-nic). My problem is that I want to enable it only for net1 and when I do it - it does nothing. And suddenly, when I enable it for net0 as well (just for testing) it starts working, blocking the traffic on both interfaces (which I don't need). What am I doing wrong?

Difficult to say without any further data - post the result of

Code:

iptables-save
grep "" /etc/pve/firewall/*

and we'll have a look.

Progratron · Mar 18, 2019

My bad

iptables-save

Code:

# Generated by iptables-save v1.6.0 on Mon Mar 18 11:05:23 2019
*filter
:INPUT ACCEPT [5:272]
:FORWARD ACCEPT [2:80]
:OUTPUT ACCEPT [0:0]
:PVEFW-Drop - [0:0]
:PVEFW-DropBroadcast - [0:0]
:PVEFW-FORWARD - [0:0]
:PVEFW-FWBR-IN - [0:0]
:PVEFW-FWBR-OUT - [0:0]
:PVEFW-HOST-IN - [0:0]
:PVEFW-HOST-OUT - [0:0]
:PVEFW-INPUT - [0:0]
:PVEFW-OUTPUT - [0:0]
:PVEFW-Reject - [0:0]
:PVEFW-SET-ACCEPT-MARK - [0:0]
:PVEFW-logflags - [0:0]
:PVEFW-reject - [0:0]
:PVEFW-smurflog - [0:0]
:PVEFW-smurfs - [0:0]
:PVEFW-tcpflags - [0:0]
:tap102i1-IN - [0:0]
:tap102i1-OUT - [0:0]
:tap202i1-IN - [0:0]
:tap202i1-OUT - [0:0]
:veth111i1-IN - [0:0]
:veth111i1-OUT - [0:0]
-A INPUT -j PVEFW-INPUT
-A FORWARD -j PVEFW-FORWARD
-A OUTPUT -j PVEFW-OUTPUT
-A PVEFW-Drop -p tcp -m tcp --dport 43 -j PVEFW-reject
-A PVEFW-Drop -j PVEFW-DropBroadcast
-A PVEFW-Drop -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
-A PVEFW-Drop -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Drop -p udp -m multiport --dports 135,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 137:139 -j DROP
-A PVEFW-Drop -p udp -m udp --sport 137 --dport 1024:65535 -j DROP
-A PVEFW-Drop -p tcp -m multiport --dports 135,139,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Drop -p udp -m udp --sport 53 -j DROP
-A PVEFW-Drop -m comment --comment "PVESIG:WDy2wbFe7jNYEyoO3QhUELZ4mIQ"
-A PVEFW-DropBroadcast -m addrtype --dst-type BROADCAST -j DROP
-A PVEFW-DropBroadcast -m addrtype --dst-type MULTICAST -j DROP
-A PVEFW-DropBroadcast -m addrtype --dst-type ANYCAST -j DROP
-A PVEFW-DropBroadcast -d 224.0.0.0/4 -j DROP
-A PVEFW-DropBroadcast -m comment --comment "PVESIG:NyjHNAtFbkH7WGLamPpdVnxHy4w"
-A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
-A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-FORWARD -m physdev --physdev-in fwln+ --physdev-is-bridged -j PVEFW-FWBR-IN
-A PVEFW-FORWARD -m physdev --physdev-out fwln+ --physdev-is-bridged -j PVEFW-FWBR-OUT
-A PVEFW-FORWARD -m comment --comment "PVESIG:qnNexOcGa+y+jebd4dAUqFSp5nw"
-A PVEFW-FWBR-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A PVEFW-FWBR-IN -m physdev --physdev-out tap102i1 --physdev-is-bridged -j tap102i1-IN
-A PVEFW-FWBR-IN -m physdev --physdev-out tap202i1 --physdev-is-bridged -j tap202i1-IN
-A PVEFW-FWBR-IN -m physdev --physdev-out veth111i1 --physdev-is-bridged -j veth111i1-IN
-A PVEFW-FWBR-IN -m comment --comment "PVESIG:vCaHziaQb+g+OnmGxeWkX9JxY3c"
-A PVEFW-FWBR-OUT -m physdev --physdev-in tap102i1 --physdev-is-bridged -j tap102i1-OUT
-A PVEFW-FWBR-OUT -m physdev --physdev-in tap202i1 --physdev-is-bridged -j tap202i1-OUT
-A PVEFW-FWBR-OUT -m physdev --physdev-in veth111i1 --physdev-is-bridged -j veth111i1-OUT
-A PVEFW-FWBR-OUT -m comment --comment "PVESIG:H9hmz4T3cXnth4o8tM9HYMrqWsc"
-A PVEFW-HOST-IN -i lo -j ACCEPT
-A PVEFW-HOST-IN -m conntrack --ctstate INVALID -j DROP
-A PVEFW-HOST-IN -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-HOST-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A PVEFW-HOST-IN -p igmp -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 8006 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 5900:5999 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 3128 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 22 -j RETURN
-A PVEFW-HOST-IN -j RETURN
-A PVEFW-HOST-IN -m comment --comment "PVESIG:uG7Hya0ggM+tHYPwTiDaovgS9Wk"
-A PVEFW-HOST-OUT -o lo -j ACCEPT
-A PVEFW-HOST-OUT -m conntrack --ctstate INVALID -j DROP
-A PVEFW-HOST-OUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-HOST-OUT -p igmp -j RETURN
-A PVEFW-HOST-OUT -j RETURN
-A PVEFW-HOST-OUT -m comment --comment "PVESIG:0zVqGNg5V7YiSxt0h+sEYJsXD+M"
-A PVEFW-INPUT -j PVEFW-HOST-IN
-A PVEFW-INPUT -m comment --comment "PVESIG:+5iMmLaxKXynOB/+5xibfx7WhFk"
-A PVEFW-OUTPUT -j PVEFW-HOST-OUT
-A PVEFW-OUTPUT -m comment --comment "PVESIG:LjHoZeSSiWAG3+2ZAyL/xuEehd0"
-A PVEFW-Reject -p tcp -m tcp --dport 43 -j PVEFW-reject
-A PVEFW-Reject -j PVEFW-DropBroadcast
-A PVEFW-Reject -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
-A PVEFW-Reject -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Reject -p udp -m multiport --dports 135,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 137:139 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --sport 137 --dport 1024:65535 -j PVEFW-reject
-A PVEFW-Reject -p tcp -m multiport --dports 135,139,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Reject -p udp -m udp --sport 53 -j DROP
-A PVEFW-Reject -m comment --comment "PVESIG:CZJnIN6rAdpu+ej59QPr9+laMUo"
-A PVEFW-SET-ACCEPT-MARK -j MARK --set-xmark 0x80000000/0x80000000
-A PVEFW-SET-ACCEPT-MARK -m comment --comment "PVESIG:Hg/OIgIwJChBUcWU8Xnjhdd2jUY"
-A PVEFW-logflags -j DROP
-A PVEFW-logflags -m comment --comment "PVESIG:MN4PH1oPZeABMuWr64RrygPfW7A"
-A PVEFW-reject -m addrtype --dst-type BROADCAST -j DROP
-A PVEFW-reject -s 224.0.0.0/4 -j DROP
-A PVEFW-reject -p icmp -j DROP
-A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
-A PVEFW-reject -p udp -j REJECT --reject-with icmp-port-unreachable
-A PVEFW-reject -p icmp -j REJECT --reject-with icmp-host-unreachable
-A PVEFW-reject -j REJECT --reject-with icmp-host-prohibited
-A PVEFW-reject -m comment --comment "PVESIG:Jlkrtle1mDdtxDeI9QaDSL++Npc"
-A PVEFW-smurflog -j DROP
-A PVEFW-smurflog -m comment --comment "PVESIG:2gfT1VMkfr0JL6OccRXTGXo+1qk"
-A PVEFW-smurfs -s 0.0.0.0/32 -j RETURN
-A PVEFW-smurfs -m addrtype --src-type BROADCAST -g PVEFW-smurflog
-A PVEFW-smurfs -s 224.0.0.0/4 -g PVEFW-smurflog
-A PVEFW-smurfs -m comment --comment "PVESIG:HssVe5QCBXd5mc9kC88749+7fag"
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags
-A PVEFW-tcpflags -m comment --comment "PVESIG:CMFojwNPqllyqD67NeI5m+bP5mo"
-A tap102i1-IN -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A tap102i1-IN -p tcp -m tcp --dport 80 -j ACCEPT
-A tap102i1-IN -p tcp -m tcp --dport 443 -j ACCEPT
-A tap102i1-IN -j PVEFW-Drop
-A tap102i1-IN -j DROP
-A tap102i1-IN -m comment --comment "PVESIG:rs2DC9fzmTAeCOMLsxlUm+BepGo"
-A tap102i1-OUT -p udp -m udp --sport 68 --dport 67 -g PVEFW-SET-ACCEPT-MARK
-A tap102i1-OUT -m mac ! --mac-source E6:F1:B4:7B:9B:A5 -j DROP
-A tap102i1-OUT -j MARK --set-xmark 0x0/0x80000000
-A tap102i1-OUT -g PVEFW-SET-ACCEPT-MARK
-A tap102i1-OUT -m comment --comment "PVESIG:gsrbVGSmfTpyfOV+EstJi+SauHw"
-A tap202i1-IN -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A tap202i1-IN -p tcp -m tcp --dport 443 -j ACCEPT
-A tap202i1-IN -j PVEFW-Drop
-A tap202i1-IN -j NFLOG --nflog-prefix  ":202:7:tap202i1-IN: policy DROP: "
-A tap202i1-IN -j DROP
-A tap202i1-IN -m comment --comment "PVESIG:E/VYfN88c7VWfr9aptiv4y6CWb4"
-A tap202i1-OUT -p udp -m udp --sport 68 --dport 67 -g PVEFW-SET-ACCEPT-MARK
-A tap202i1-OUT -m mac ! --mac-source 96:70:C4:CD:DA:53 -j DROP
-A tap202i1-OUT -j MARK --set-xmark 0x0/0x80000000
-A tap202i1-OUT -g PVEFW-SET-ACCEPT-MARK
-A tap202i1-OUT -m comment --comment "PVESIG:N+/b+vuscAU0TN/FaGk/vsFimno"
-A veth111i1-IN -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A veth111i1-IN -j PVEFW-Drop
-A veth111i1-IN -j DROP
-A veth111i1-IN -m comment --comment "PVESIG:tk4G7z9JUa1GodI1exNxelwvvgk"
-A veth111i1-OUT -p udp -m udp --sport 68 --dport 67 -g PVEFW-SET-ACCEPT-MARK
-A veth111i1-OUT -m mac ! --mac-source DE:3A:04:96:48:EC -j DROP
-A veth111i1-OUT -j MARK --set-xmark 0x0/0x80000000
-A veth111i1-OUT -g PVEFW-SET-ACCEPT-MARK
-A veth111i1-OUT -m comment --comment "PVESIG:C5CuiZ7ycXf80HC3VH401PClHd4"
COMMIT
# Completed on Mon Mar 18 11:05:23 2019
# Generated by iptables-save v1.6.0 on Mon Mar 18 11:05:23 2019
*nat
:PREROUTING ACCEPT [1011:65658]
:INPUT ACCEPT [186:10693]
:OUTPUT ACCEPT [32:2758]
:POSTROUTING ACCEPT [790:53143]
-A POSTROUTING -s 10.10.20.0/24 ! -d 192.168.0.0/21 -o enp2s0 -j MASQUERADE
COMMIT
# Completed on Mon Mar 18 11:05:23 2019

grep "" /etc/pve/firewall/*

Code:

/etc/pve/firewall/102.fw:[OPTIONS]
/etc/pve/firewall/102.fw:
/etc/pve/firewall/102.fw:enable: 1
/etc/pve/firewall/102.fw:
/etc/pve/firewall/102.fw:[RULES]
/etc/pve/firewall/102.fw:
/etc/pve/firewall/102.fw:IN HTTP(ACCEPT) -i net1
/etc/pve/firewall/102.fw:IN HTTPS(ACCEPT) -i net1
/etc/pve/firewall/102.fw:
/etc/pve/firewall/111.fw:[OPTIONS]
/etc/pve/firewall/111.fw:
/etc/pve/firewall/111.fw:enable: 1
/etc/pve/firewall/111.fw:policy_in: DROP
/etc/pve/firewall/111.fw:
/etc/pve/firewall/200.fw:[OPTIONS]
/etc/pve/firewall/200.fw:
/etc/pve/firewall/200.fw:enable: 0
/etc/pve/firewall/200.fw:
/etc/pve/firewall/202.fw:[OPTIONS]
/etc/pve/firewall/202.fw:
/etc/pve/firewall/202.fw:log_level_in: debug
/etc/pve/firewall/202.fw:enable: 1
/etc/pve/firewall/202.fw:
/etc/pve/firewall/202.fw:[RULES]
/etc/pve/firewall/202.fw:
/etc/pve/firewall/202.fw:IN HTTPS(ACCEPT) -i net1
/etc/pve/firewall/202.fw:
/etc/pve/firewall/cluster.fw:[OPTIONS]
/etc/pve/firewall/cluster.fw:
/etc/pve/firewall/cluster.fw:enable: 1
/etc/pve/firewall/cluster.fw:policy_in: ACCEPT
/etc/pve/firewall/cluster.fw:

Richard · Mar 18, 2019

I've configured the setting you have in a test environment and it works as expected.
Give an example what in particular does not work as expected in your case (source, destination, port).

Progratron · Mar 18, 2019

Thanks for the prompt reply. Well under current settings I am expecting all the incoming traffic to net1 of VM111 being dropped. This doesn't happen though:

10.10.20.111 - net0
PUBLIC_IP_NET1 - net1

Code:

[~]$ nmap -p 443 10.10.20.111
Starting Nmap 7.70 ( https://nmap.org ) at 2019-03-18 12:22 CET
Nmap scan report for 10.10.20.111
Host is up (0.041s latency).

PORT    STATE SERVICE
443/tcp open  https

Nmap done: 1 IP address (1 host up) scanned in 0.17 seconds
[~]$ nmap -p 443 PUBLIC_IP_NET1
Starting Nmap 7.70 ( https://nmap.org ) at 2019-03-18 12:22 CET
Nmap scan report for XXX (PUBLIC_IP_NET1)
Host is up (0.040s latency).

PORT    STATE SERVICE
443/tcp open  https

Nmap done: 1 IP address (1 host up) scanned in 0.22 seconds
[~]$

Search

Search

Firewall not working on a single NIC out of two?

Progratron

Active Member

Richard

Renowned Member

Progratron

Active Member

Richard

Renowned Member

Progratron

Active Member