Turning on the PVE Firewall stops VM / LXC connectivity

F

FastLaneJB

Renowned Member
Feb 3, 2012
90
6
73
Hi all,

I've had the firewall working previously on a different host however on a new install recently I'm having problems with it and at a bit of a loss as to what the issue is.

If I turn the firewall on in the Datacentre all communication with KVM and LXC machines running on the host just stops. I cannot connect to them, I cannot ping them. The host itself is fine and even making changes in the hosts firewall to allow a new port works when I test it.

Only one of the LXC's has the firewall enabled at this point but even with that off nothing works at all. I've put the output of a couple of the commands I've seen in other threads that hopefully will help. Any ideas?

Compile

Code: 
ipset cmdlist:
exists PVEFW-0-management-v4 (hwAfqIGn9k7XQSlehQ9eeyB5+uM)
        create PVEFW-0-management-v4 hash:net family inet hashsize 64 maxelem 64
        add PVEFW-0-management-v4 172.28.28.0/23
exists PVEFW-0-management-v6 (H5WO/Pkuyz4e7OLB2uiMpG0Bsn0)
        create PVEFW-0-management-v6 hash:net family inet6 hashsize 64 maxelem 64

iptables cmdlist:
exists PVEFW-Drop (WDy2wbFe7jNYEyoO3QhUELZ4mIQ)
        -A PVEFW-Drop -p tcp --dport 43 -j PVEFW-reject
        -A PVEFW-Drop  -j PVEFW-DropBroadcast
        -A PVEFW-Drop -p icmp -m icmp --icmp-type fragmentation-needed -j ACCEPT
        -A PVEFW-Drop -p icmp -m icmp --icmp-type time-exceeded -j ACCEPT
        -A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-Drop -p udp --match multiport --dports 135,445 -j DROP
        -A PVEFW-Drop -p udp --dport 137:139 -j DROP
        -A PVEFW-Drop -p udp --sport 137 --dport 1024:65535 -j DROP
        -A PVEFW-Drop -p tcp --match multiport --dports 135,139,445 -j DROP
        -A PVEFW-Drop -p udp --dport 1900 -j DROP
        -A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
        -A PVEFW-Drop -p udp --sport 53 -j DROP
exists PVEFW-DropBroadcast (NyjHNAtFbkH7WGLamPpdVnxHy4w)
        -A PVEFW-DropBroadcast -m addrtype --dst-type BROADCAST -j DROP
        -A PVEFW-DropBroadcast -m addrtype --dst-type MULTICAST -j DROP
        -A PVEFW-DropBroadcast -m addrtype --dst-type ANYCAST -j DROP
        -A PVEFW-DropBroadcast -d 224.0.0.0/4 -j DROP
exists PVEFW-FORWARD (qnNexOcGa+y+jebd4dAUqFSp5nw)
        -A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
        -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-in fwln+ -j PVEFW-FWBR-IN
        -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-out fwln+ -j PVEFW-FWBR-OUT
exists PVEFW-FWBR-IN (QLpD+E2027zOYZ2nkV6qgZBluA8)
        -A PVEFW-FWBR-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
        -A PVEFW-FWBR-IN -m physdev --physdev-is-bridged --physdev-out veth8000i0 -j veth8000i0-IN
exists PVEFW-FWBR-OUT (NWXsCfKmci8gqZK9tPB4ZpwPFpQ)
        -A PVEFW-FWBR-OUT -m physdev --physdev-is-bridged --physdev-in veth8000i0 -j veth8000i0-OUT
exists PVEFW-INPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-OUTPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-Reject (CZJnIN6rAdpu+ej59QPr9+laMUo)
        -A PVEFW-Reject -p tcp --dport 43 -j PVEFW-reject
        -A PVEFW-Reject  -j PVEFW-DropBroadcast
        -A PVEFW-Reject -p icmp -m icmp --icmp-type fragmentation-needed -j ACCEPT
        -A PVEFW-Reject -p icmp -m icmp --icmp-type time-exceeded -j ACCEPT
        -A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-Reject -p udp --match multiport --dports 135,445 -j PVEFW-reject
        -A PVEFW-Reject -p udp --dport 137:139 -j PVEFW-reject
        -A PVEFW-Reject -p udp --sport 137 --dport 1024:65535 -j PVEFW-reject
        -A PVEFW-Reject -p tcp --match multiport --dports 135,139,445 -j PVEFW-reject
        -A PVEFW-Reject -p udp --dport 1900 -j DROP
        -A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
        -A PVEFW-Reject -p udp --sport 53 -j DROP
exists PVEFW-SET-ACCEPT-MARK (Hg/OIgIwJChBUcWU8Xnjhdd2jUY)
        -A PVEFW-SET-ACCEPT-MARK  -j MARK --set-mark 0x80000000/0x80000000
exists PVEFW-logflags (MN4PH1oPZeABMuWr64RrygPfW7A)
        -A PVEFW-logflags  -j DROP
exists PVEFW-reject (Jlkrtle1mDdtxDeI9QaDSL++Npc)
        -A PVEFW-reject -m addrtype --dst-type BROADCAST -j DROP
        -A PVEFW-reject -s 224.0.0.0/4 -j DROP
        -A PVEFW-reject -p icmp -j DROP
        -A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
        -A PVEFW-reject -p udp -j REJECT --reject-with icmp-port-unreachable
        -A PVEFW-reject -p icmp -j REJECT --reject-with icmp-host-unreachable
        -A PVEFW-reject  -j REJECT --reject-with icmp-host-prohibited
exists PVEFW-smurflog (2gfT1VMkfr0JL6OccRXTGXo+1qk)
        -A PVEFW-smurflog  -j DROP
exists PVEFW-smurfs (HssVe5QCBXd5mc9kC88749+7fag)
        -A PVEFW-smurfs -s 0.0.0.0/32 -j RETURN
        -A PVEFW-smurfs -m addrtype --src-type BROADCAST -g PVEFW-smurflog
        -A PVEFW-smurfs -s 224.0.0.0/4 -g PVEFW-smurflog
exists PVEFW-tcpflags (CMFojwNPqllyqD67NeI5m+bP5mo)
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags
exists veth8000i0-IN (GRGMA98bAX1ahiRLM9WW908KSDE)
        -A veth8000i0-IN -p udp --sport 67 --dport 68 -j ACCEPT
        -A veth8000i0-IN -j PVEFW-Drop
        -A veth8000i0-IN -j DROP
exists veth8000i0-OUT (lLrOSo1yg5mTXUI+Zva3sfqiteU)
        -A veth8000i0-OUT -p udp --sport 68 --dport 67 -g PVEFW-SET-ACCEPT-MARK
        -A veth8000i0-OUT -m mac ! --mac-source 62:6F:D7:C6:E7:7F -j DROP
        -A veth8000i0-OUT -j MARK --set-mark 0x00000000/0x80000000
        -A veth8000i0-OUT  -g PVEFW-SET-ACCEPT-MARK

ip6tables cmdlist:
exists PVEFW-Drop (Jb79Uw7z1vZglIcV7QXA5uY/nbk)
        -A PVEFW-Drop -p tcp --dport 43 -j PVEFW-reject
        -A PVEFW-Drop  -j PVEFW-DropBroadcast
        -A PVEFW-Drop -p icmpv6 -m icmpv6 --icmpv6-type destination-unreachable -j ACCEPT
        -A PVEFW-Drop -p icmpv6 -m icmpv6 --icmpv6-type time-exceeded -j ACCEPT
        -A PVEFW-Drop -p icmpv6 -m icmpv6 --icmpv6-type packet-too-big -j ACCEPT
        -A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-Drop -p udp --match multiport --dports 135,445 -j DROP
        -A PVEFW-Drop -p udp --dport 137:139 -j DROP
        -A PVEFW-Drop -p udp --sport 137 --dport 1024:65535 -j DROP
        -A PVEFW-Drop -p tcp --match multiport --dports 135,139,445 -j DROP
        -A PVEFW-Drop -p udp --dport 1900 -j DROP
        -A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
        -A PVEFW-Drop -p udp --sport 53 -j DROP
exists PVEFW-DropBroadcast (8Krk5Nh8pDZOOc7BQAbM6PlyFSU)
        -A PVEFW-DropBroadcast -d ff00::/8 -j DROP
exists PVEFW-FORWARD (qnNexOcGa+y+jebd4dAUqFSp5nw)
        -A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
        -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-in fwln+ -j PVEFW-FWBR-IN
        -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-out fwln+ -j PVEFW-FWBR-OUT
exists PVEFW-FWBR-IN (/8xZDAArcEGGxr2UiUCyd8ymnHk)
        -A PVEFW-FWBR-IN -m physdev --physdev-is-bridged --physdev-out veth8000i0 -j veth8000i0-IN
exists PVEFW-FWBR-OUT (NWXsCfKmci8gqZK9tPB4ZpwPFpQ)
        -A PVEFW-FWBR-OUT -m physdev --physdev-is-bridged --physdev-in veth8000i0 -j veth8000i0-OUT
exists PVEFW-INPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-OUTPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-Reject (aL1nrxJk/u3XmTb3Am2eaM/3yCM)
        -A PVEFW-Reject -p tcp --dport 43 -j PVEFW-reject
        -A PVEFW-Reject  -j PVEFW-DropBroadcast
        -A PVEFW-Reject -p icmpv6 -m icmpv6 --icmpv6-type destination-unreachable -j ACCEPT
        -A PVEFW-Reject -p icmpv6 -m icmpv6 --icmpv6-type time-exceeded -j ACCEPT
        -A PVEFW-Reject -p icmpv6 -m icmpv6 --icmpv6-type packet-too-big -j ACCEPT
        -A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-Reject -p udp --match multiport --dports 135,445 -j PVEFW-reject
        -A PVEFW-Reject -p udp --dport 137:139 -j PVEFW-reject
        -A PVEFW-Reject -p udp --sport 137 --dport 1024:65535 -j PVEFW-reject
        -A PVEFW-Reject -p tcp --match multiport --dports 135,139,445 -j PVEFW-reject
        -A PVEFW-Reject -p udp --dport 1900 -j DROP
        -A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
        -A PVEFW-Reject -p udp --sport 53 -j DROP
exists PVEFW-SET-ACCEPT-MARK (Hg/OIgIwJChBUcWU8Xnjhdd2jUY)
        -A PVEFW-SET-ACCEPT-MARK  -j MARK --set-mark 0x80000000/0x80000000
exists PVEFW-logflags (MN4PH1oPZeABMuWr64RrygPfW7A)
        -A PVEFW-logflags  -j DROP
exists PVEFW-reject (etEECUYcgUdzuuO+LDP83pu0S8Y)
        -A PVEFW-reject -p icmpv6 -j DROP
        -A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
        -A PVEFW-reject -p udp -j REJECT --reject-with icmp6-port-unreachable
        -A PVEFW-reject  -j REJECT --reject-with icmp6-adm-prohibited
exists PVEFW-tcpflags (CMFojwNPqllyqD67NeI5m+bP5mo)
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags
exists veth8000i0-IN (yXvx5rBp1B2l2QNaSsem99/pgO0)
        -A veth8000i0-IN -p udp --sport 547 --dport 546 -j ACCEPT
        -A veth8000i0-IN -p icmpv6 --icmpv6-type router-solicitation -j ACCEPT
        -A veth8000i0-IN -p icmpv6 --icmpv6-type router-advertisement -j ACCEPT
        -A veth8000i0-IN -p icmpv6 --icmpv6-type neighbor-solicitation -j ACCEPT
        -A veth8000i0-IN -p icmpv6 --icmpv6-type neighbor-advertisement -j ACCEPT
        -A veth8000i0-IN -j PVEFW-Drop
        -A veth8000i0-IN -j DROP
exists veth8000i0-OUT (byg2wWHpZ47PVOpXqFSWCjJWXXw)
        -A veth8000i0-OUT -p udp --sport 546 --dport 547 -g PVEFW-SET-ACCEPT-MARK
        -A veth8000i0-OUT -m mac ! --mac-source 62:6F:D7:C6:E7:7F -j DROP
        -A veth8000i0-OUT -p icmpv6 --icmpv6-type router-advertisement -j DROP
        -A veth8000i0-OUT -j MARK --set-mark 0x00000000/0x80000000
        -A veth8000i0-OUT -p icmpv6 --icmpv6-type router-solicitation -g PVEFW-SET-ACCEPT-MARK
        -A veth8000i0-OUT -p icmpv6 --icmpv6-type neighbor-solicitation -g PVEFW-SET-ACCEPT-MARK
        -A veth8000i0-OUT -p icmpv6 --icmpv6-type neighbor-advertisement -g PVEFW-SET-ACCEPT-MARK
        -A veth8000i0-OUT  -g PVEFW-SET-ACCEPT-MARK

ebtables cmdlist:
exists PVEFW-FORWARD (ULtZ6lqjrD/jAKLY+OZo3BbXs9k)
        -A PVEFW-FORWARD -p IPv4 -j ACCEPT
        -A PVEFW-FORWARD -p IPv6 -j ACCEPT
        -A PVEFW-FORWARD -o fwln+ -j PVEFW-FWBR-OUT
exists PVEFW-FWBR-OUT (uOWRlE0blWG+U/FHCClItwJY9Z0)
        -A PVEFW-FWBR-OUT -i veth8000i0 -j veth8000i0-OUT
exists veth8000i0-OUT (sJsvisMrpRBJkB0a3gFHZKTB8qs)
        -A veth8000i0-OUT -s ! 62:6f:d7:c6:e7:7f -j DROP
        -A veth8000i0-OUT -j ACCEPT
ignore FORWARD (zuQi5YOvmMWiM9zohnQw/qWemOA)
ignore INPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore OUTPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
no changes

IP Tables Save

Code: 
# Generated by iptables-save v1.6.0 on Wed Jul  3 17:52:23 2019
*nat
:PREROUTING ACCEPT [2732:747262]
:INPUT ACCEPT [62:6536]
:OUTPUT ACCEPT [64:5057]
:POSTROUTING ACCEPT [1966:408391]
COMMIT
# Completed on Wed Jul  3 17:52:23 2019
# Generated by iptables-save v1.6.0 on Wed Jul  3 17:52:23 2019
*mangle
:PREROUTING ACCEPT [3309584:2937130602]
:INPUT ACCEPT [30210:50276370]
:FORWARD ACCEPT [3302273:2885499833]
:OUTPUT ACCEPT [16218:7448926]
:POSTROUTING ACCEPT [3317349:2892893145]
COMMIT
# Completed on Wed Jul  3 17:52:23 2019
# Generated by iptables-save v1.6.0 on Wed Jul  3 17:52:23 2019
*filter
:INPUT ACCEPT [320:83264]
:FORWARD ACCEPT [25657:34015561]
:OUTPUT ACCEPT [226:137757]
:PVEFW-Drop - [0:0]
:PVEFW-DropBroadcast - [0:0]
:PVEFW-FORWARD - [0:0]
:PVEFW-FWBR-IN - [0:0]
:PVEFW-FWBR-OUT - [0:0]
:PVEFW-INPUT - [0:0]
:PVEFW-OUTPUT - [0:0]
:PVEFW-Reject - [0:0]
:PVEFW-SET-ACCEPT-MARK - [0:0]
:PVEFW-logflags - [0:0]
:PVEFW-reject - [0:0]
:PVEFW-smurflog - [0:0]
:PVEFW-smurfs - [0:0]
:PVEFW-tcpflags - [0:0]
:veth8000i0-IN - [0:0]
:veth8000i0-OUT - [0:0]
-A INPUT -j PVEFW-INPUT
-A FORWARD -j PVEFW-FORWARD
-A OUTPUT -j PVEFW-OUTPUT
-A PVEFW-Drop -p tcp -m tcp --dport 43 -j PVEFW-reject
-A PVEFW-Drop -j PVEFW-DropBroadcast
-A PVEFW-Drop -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
-A PVEFW-Drop -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Drop -p udp -m multiport --dports 135,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 137:139 -j DROP
-A PVEFW-Drop -p udp -m udp --sport 137 --dport 1024:65535 -j DROP
-A PVEFW-Drop -p tcp -m multiport --dports 135,139,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Drop -p udp -m udp --sport 53 -j DROP
-A PVEFW-Drop -m comment --comment "PVESIG:WDy2wbFe7jNYEyoO3QhUELZ4mIQ"
-A PVEFW-DropBroadcast -m addrtype --dst-type BROADCAST -j DROP
-A PVEFW-DropBroadcast -m addrtype --dst-type MULTICAST -j DROP
-A PVEFW-DropBroadcast -m addrtype --dst-type ANYCAST -j DROP
-A PVEFW-DropBroadcast -d 224.0.0.0/4 -j DROP
-A PVEFW-DropBroadcast -m comment --comment "PVESIG:NyjHNAtFbkH7WGLamPpdVnxHy4w"
-A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
-A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-FORWARD -m physdev --physdev-in fwln+ --physdev-is-bridged -j PVEFW-FWBR-IN
-A PVEFW-FORWARD -m physdev --physdev-out fwln+ --physdev-is-bridged -j PVEFW-FWBR-OUT
-A PVEFW-FORWARD -m comment --comment "PVESIG:qnNexOcGa+y+jebd4dAUqFSp5nw"
-A PVEFW-FWBR-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A PVEFW-FWBR-IN -m physdev --physdev-out veth8000i0 --physdev-is-bridged -j veth8000i0-IN
-A PVEFW-FWBR-IN -m comment --comment "PVESIG:QLpD+E2027zOYZ2nkV6qgZBluA8"
-A PVEFW-FWBR-OUT -m physdev --physdev-in veth8000i0 --physdev-is-bridged -j veth8000i0-OUT
-A PVEFW-FWBR-OUT -m comment --comment "PVESIG:NWXsCfKmci8gqZK9tPB4ZpwPFpQ"
-A PVEFW-INPUT -m comment --comment "PVESIG:2jmj7l5rSw0yVb/vlWAYkK/YBwk"
-A PVEFW-OUTPUT -m comment --comment "PVESIG:2jmj7l5rSw0yVb/vlWAYkK/YBwk"
-A PVEFW-Reject -p tcp -m tcp --dport 43 -j PVEFW-reject
-A PVEFW-Reject -j PVEFW-DropBroadcast
-A PVEFW-Reject -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
-A PVEFW-Reject -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Reject -p udp -m multiport --dports 135,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 137:139 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --sport 137 --dport 1024:65535 -j PVEFW-reject
-A PVEFW-Reject -p tcp -m multiport --dports 135,139,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Reject -p udp -m udp --sport 53 -j DROP
-A PVEFW-Reject -m comment --comment "PVESIG:CZJnIN6rAdpu+ej59QPr9+laMUo"
-A PVEFW-SET-ACCEPT-MARK -j MARK --set-xmark 0x80000000/0x80000000
-A PVEFW-SET-ACCEPT-MARK -m comment --comment "PVESIG:Hg/OIgIwJChBUcWU8Xnjhdd2jUY"
-A PVEFW-logflags -j DROP
-A PVEFW-logflags -m comment --comment "PVESIG:MN4PH1oPZeABMuWr64RrygPfW7A"
-A PVEFW-reject -m addrtype --dst-type BROADCAST -j DROP
-A PVEFW-reject -s 224.0.0.0/4 -j DROP
-A PVEFW-reject -p icmp -j DROP
-A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
-A PVEFW-reject -p udp -j REJECT --reject-with icmp-port-unreachable
-A PVEFW-reject -p icmp -j REJECT --reject-with icmp-host-unreachable
-A PVEFW-reject -j REJECT --reject-with icmp-host-prohibited
-A PVEFW-reject -m comment --comment "PVESIG:Jlkrtle1mDdtxDeI9QaDSL++Npc"
-A PVEFW-smurflog -j DROP
-A PVEFW-smurflog -m comment --comment "PVESIG:2gfT1VMkfr0JL6OccRXTGXo+1qk"
-A PVEFW-smurfs -s 0.0.0.0/32 -j RETURN
-A PVEFW-smurfs -m addrtype --src-type BROADCAST -g PVEFW-smurflog
-A PVEFW-smurfs -s 224.0.0.0/4 -g PVEFW-smurflog
-A PVEFW-smurfs -m comment --comment "PVESIG:HssVe5QCBXd5mc9kC88749+7fag"
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags
-A PVEFW-tcpflags -m comment --comment "PVESIG:CMFojwNPqllyqD67NeI5m+bP5mo"
-A veth8000i0-IN -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A veth8000i0-IN -j PVEFW-Drop
-A veth8000i0-IN -j DROP
-A veth8000i0-IN -m comment --comment "PVESIG:GRGMA98bAX1ahiRLM9WW908KSDE"
-A veth8000i0-OUT -p udp -m udp --sport 68 --dport 67 -g PVEFW-SET-ACCEPT-MARK
-A veth8000i0-OUT -m mac ! --mac-source 62:6F:D7:C6:E7:7F -j DROP
-A veth8000i0-OUT -j MARK --set-xmark 0x0/0x80000000
-A veth8000i0-OUT -g PVEFW-SET-ACCEPT-MARK
-A veth8000i0-OUT -m comment --comment "PVESIG:lLrOSo1yg5mTXUI+Zva3sfqiteU"
COMMIT
# Completed on Wed Jul  3 17:52:23 2019
 
Hi,

I'm experience the same problem. If I disable the pve-firewall doesn't work until I reboot the host. It's very weird.

Could you sort out the problem?

Best regards.
 
Ah I don’t have to reboot in my case. Switching it off brings the connectivity back.

I’ve of course got a firewall across VLAN’s but I liked that I could firewall inside subnet so one machine in a DMZ couldn’t communicate with another unless I allowed it. The firewall cannot help me there as the traffic doesn’t touch it and I’d rather do it this way than VM / LXC OS firewall rules because I can share across machines.
 
As an update but made a new VM recently and see that the Firewall is ticked on by default on the VM NIC. Even with the Datacentre firewall setting switched off this breaks all connectivity to the VM until I switch the firewall off.

Is it just lots of people don’t use the firewall or something strange going on because my Proxmox is pretty vanilla and out of the box? Just seems strange this hasn’t been noticed by many others?
 
FastLaneJB said:
As an update but made a new VM recently and see that the Firewall is ticked on by default on the VM NIC. Even with the Datacentre firewall setting switched off this breaks all connectivity to the VM until I switch the firewall off.

Is it just lots of people don’t use the firewall or something strange going on because my Proxmox is pretty vanilla and out of the box? Just seems strange this hasn’t been noticed by many others?
Click to expand...

could you provide 'pve-firewall status' and 'pve-firewall compile' output both before and after?

the first thing the firewall does on updating the rules is check whether it is enabled on the datacenter/cluster level, and if not, remove all the custom chains and exit.. I just noticed that ebtables rules are not cleared in that situation, so maybe you could try enabling the firewall again, but disabling ebtables (also on the cluster/datacenter level) to see if those leftover rules are the culprit?
 
Hi Fabian,

No problems at all. This is a long post but hopefully has the info you need but let me know if you need me to try anything else.

I will of course wipe this server and reload it with v6.0 when that comes out but suspect this might be an issue there so hopefully solving that now solves it for 6.0 as well.

And as always love your work, keep it up :)

All off

pve-firewall status

Code: 
root@kt-pve-01:~# pve-firewall status
Status: disabled/running

pve-firewall compile

Code: 
root@kt-pve-01:~# pve-firewall compile
ipset cmdlist:

iptables cmdlist:

ip6tables cmdlist:

ebtables cmdlist:
ignore BROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore FORWARD (zuQi5YOvmMWiM9zohnQw/qWemOA)
ignore INPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore OUTPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore POSTROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore PREROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
delete PVEFW-FORWARD (ULtZ6lqjrD/jAKLY+OZo3BbXs9k)
delete PVEFW-FWBR-OUT (uOWRlE0blWG+U/FHCClItwJY9Z0)
delete veth8000i0-OUT (sJsvisMrpRBJkB0a3gFHZKTB8qs)
no changes
firewall disabled

Firewall on at Datacentre level - etables Yes

This is off at host and also VM's but by this point network connectivity is now broken

pve-firewall status

Code: 
root@kt-pve-01:~# pve-firewall status
Status: enabled/running

pve-firewall compile

Code: 
root@kt-pve-01:~# pve-firewall compile
ipset cmdlist:

iptables cmdlist:

ip6tables cmdlist:

ebtables cmdlist:
ignore BROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore FORWARD (zuQi5YOvmMWiM9zohnQw/qWemOA)
ignore INPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore OUTPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore POSTROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore PREROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
delete PVEFW-FORWARD (ULtZ6lqjrD/jAKLY+OZo3BbXs9k)
delete PVEFW-FWBR-OUT (uOWRlE0blWG+U/FHCClItwJY9Z0)
delete veth8000i0-OUT (sJsvisMrpRBJkB0a3gFHZKTB8qs)
no changes
firewall disabled
root@kt-pve-01:~# pve-firewall status
Status: enabled/running
root@kt-pve-01:~# pve-firewall compile
ipset cmdlist:
exists PVEFW-0-management-v4 (hwAfqIGn9k7XQSlehQ9eeyB5+uM)
        create PVEFW-0-management-v4 hash:net family inet hashsize 64 maxelem 64
        add PVEFW-0-management-v4 172.28.28.0/23
exists PVEFW-0-management-v6 (H5WO/Pkuyz4e7OLB2uiMpG0Bsn0)
        create PVEFW-0-management-v6 hash:net family inet6 hashsize 64 maxelem 64

iptables cmdlist:
exists PVEFW-Drop (WDy2wbFe7jNYEyoO3QhUELZ4mIQ)
        -A PVEFW-Drop -p tcp --dport 43 -j PVEFW-reject
        -A PVEFW-Drop  -j PVEFW-DropBroadcast
        -A PVEFW-Drop -p icmp -m icmp --icmp-type fragmentation-needed -j ACCEPT
        -A PVEFW-Drop -p icmp -m icmp --icmp-type time-exceeded -j ACCEPT
        -A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-Drop -p udp --match multiport --dports 135,445 -j DROP
        -A PVEFW-Drop -p udp --dport 137:139 -j DROP
        -A PVEFW-Drop -p udp --sport 137 --dport 1024:65535 -j DROP
        -A PVEFW-Drop -p tcp --match multiport --dports 135,139,445 -j DROP
        -A PVEFW-Drop -p udp --dport 1900 -j DROP
        -A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
        -A PVEFW-Drop -p udp --sport 53 -j DROP
exists PVEFW-DropBroadcast (NyjHNAtFbkH7WGLamPpdVnxHy4w)
        -A PVEFW-DropBroadcast -m addrtype --dst-type BROADCAST -j DROP
        -A PVEFW-DropBroadcast -m addrtype --dst-type MULTICAST -j DROP
        -A PVEFW-DropBroadcast -m addrtype --dst-type ANYCAST -j DROP
        -A PVEFW-DropBroadcast -d 224.0.0.0/4 -j DROP
exists PVEFW-FORWARD (qnNexOcGa+y+jebd4dAUqFSp5nw)
        -A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
        -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-in fwln+ -j PVEFW-FWBR-IN
        -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-out fwln+ -j PVEFW-FWBR-OUT
exists PVEFW-FWBR-IN (Ijl7/xz0DD7LF91MlLCz0ybZBE0)
        -A PVEFW-FWBR-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
exists PVEFW-FWBR-OUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-INPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-OUTPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-Reject (CZJnIN6rAdpu+ej59QPr9+laMUo)
        -A PVEFW-Reject -p tcp --dport 43 -j PVEFW-reject
        -A PVEFW-Reject  -j PVEFW-DropBroadcast
        -A PVEFW-Reject -p icmp -m icmp --icmp-type fragmentation-needed -j ACCEPT
        -A PVEFW-Reject -p icmp -m icmp --icmp-type time-exceeded -j ACCEPT
        -A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-Reject -p udp --match multiport --dports 135,445 -j PVEFW-reject
        -A PVEFW-Reject -p udp --dport 137:139 -j PVEFW-reject
        -A PVEFW-Reject -p udp --sport 137 --dport 1024:65535 -j PVEFW-reject
        -A PVEFW-Reject -p tcp --match multiport --dports 135,139,445 -j PVEFW-reject
        -A PVEFW-Reject -p udp --dport 1900 -j DROP
        -A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
        -A PVEFW-Reject -p udp --sport 53 -j DROP
exists PVEFW-SET-ACCEPT-MARK (Hg/OIgIwJChBUcWU8Xnjhdd2jUY)
        -A PVEFW-SET-ACCEPT-MARK  -j MARK --set-mark 0x80000000/0x80000000
exists PVEFW-logflags (MN4PH1oPZeABMuWr64RrygPfW7A)
        -A PVEFW-logflags  -j DROP
exists PVEFW-reject (Jlkrtle1mDdtxDeI9QaDSL++Npc)
        -A PVEFW-reject -m addrtype --dst-type BROADCAST -j DROP
        -A PVEFW-reject -s 224.0.0.0/4 -j DROP
        -A PVEFW-reject -p icmp -j DROP
        -A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
        -A PVEFW-reject -p udp -j REJECT --reject-with icmp-port-unreachable
        -A PVEFW-reject -p icmp -j REJECT --reject-with icmp-host-unreachable
        -A PVEFW-reject  -j REJECT --reject-with icmp-host-prohibited
exists PVEFW-smurflog (2gfT1VMkfr0JL6OccRXTGXo+1qk)
        -A PVEFW-smurflog  -j DROP
exists PVEFW-smurfs (HssVe5QCBXd5mc9kC88749+7fag)
        -A PVEFW-smurfs -s 0.0.0.0/32 -j RETURN
        -A PVEFW-smurfs -m addrtype --src-type BROADCAST -g PVEFW-smurflog
        -A PVEFW-smurfs -s 224.0.0.0/4 -g PVEFW-smurflog
exists PVEFW-tcpflags (CMFojwNPqllyqD67NeI5m+bP5mo)
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags

ip6tables cmdlist:
exists PVEFW-Drop (Jb79Uw7z1vZglIcV7QXA5uY/nbk)
        -A PVEFW-Drop -p tcp --dport 43 -j PVEFW-reject
        -A PVEFW-Drop  -j PVEFW-DropBroadcast
        -A PVEFW-Drop -p icmpv6 -m icmpv6 --icmpv6-type destination-unreachable -j ACCEPT
        -A PVEFW-Drop -p icmpv6 -m icmpv6 --icmpv6-type time-exceeded -j ACCEPT
        -A PVEFW-Drop -p icmpv6 -m icmpv6 --icmpv6-type packet-too-big -j ACCEPT
        -A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-Drop -p udp --match multiport --dports 135,445 -j DROP
        -A PVEFW-Drop -p udp --dport 137:139 -j DROP
        -A PVEFW-Drop -p udp --sport 137 --dport 1024:65535 -j DROP
        -A PVEFW-Drop -p tcp --match multiport --dports 135,139,445 -j DROP
        -A PVEFW-Drop -p udp --dport 1900 -j DROP
        -A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
        -A PVEFW-Drop -p udp --sport 53 -j DROP
exists PVEFW-DropBroadcast (8Krk5Nh8pDZOOc7BQAbM6PlyFSU)
        -A PVEFW-DropBroadcast -d ff00::/8 -j DROP
exists PVEFW-FORWARD (qnNexOcGa+y+jebd4dAUqFSp5nw)
        -A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
        -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-in fwln+ -j PVEFW-FWBR-IN
        -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-out fwln+ -j PVEFW-FWBR-OUT
exists PVEFW-FWBR-IN (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-FWBR-OUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-INPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-OUTPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-Reject (aL1nrxJk/u3XmTb3Am2eaM/3yCM)
        -A PVEFW-Reject -p tcp --dport 43 -j PVEFW-reject
        -A PVEFW-Reject  -j PVEFW-DropBroadcast
        -A PVEFW-Reject -p icmpv6 -m icmpv6 --icmpv6-type destination-unreachable -j ACCEPT
        -A PVEFW-Reject -p icmpv6 -m icmpv6 --icmpv6-type time-exceeded -j ACCEPT
        -A PVEFW-Reject -p icmpv6 -m icmpv6 --icmpv6-type packet-too-big -j ACCEPT
        -A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-Reject -p udp --match multiport --dports 135,445 -j PVEFW-reject
        -A PVEFW-Reject -p udp --dport 137:139 -j PVEFW-reject
        -A PVEFW-Reject -p udp --sport 137 --dport 1024:65535 -j PVEFW-reject
        -A PVEFW-Reject -p tcp --match multiport --dports 135,139,445 -j PVEFW-reject
        -A PVEFW-Reject -p udp --dport 1900 -j DROP
        -A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
        -A PVEFW-Reject -p udp --sport 53 -j DROP
exists PVEFW-SET-ACCEPT-MARK (Hg/OIgIwJChBUcWU8Xnjhdd2jUY)
        -A PVEFW-SET-ACCEPT-MARK  -j MARK --set-mark 0x80000000/0x80000000
exists PVEFW-logflags (MN4PH1oPZeABMuWr64RrygPfW7A)
        -A PVEFW-logflags  -j DROP
exists PVEFW-reject (etEECUYcgUdzuuO+LDP83pu0S8Y)
        -A PVEFW-reject -p icmpv6 -j DROP
        -A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
        -A PVEFW-reject -p udp -j REJECT --reject-with icmp6-port-unreachable
        -A PVEFW-reject  -j REJECT --reject-with icmp6-adm-prohibited
exists PVEFW-tcpflags (CMFojwNPqllyqD67NeI5m+bP5mo)
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags

ebtables cmdlist:
exists PVEFW-FORWARD (ULtZ6lqjrD/jAKLY+OZo3BbXs9k)
        -A PVEFW-FORWARD -p IPv4 -j ACCEPT
        -A PVEFW-FORWARD -p IPv6 -j ACCEPT
        -A PVEFW-FORWARD -o fwln+ -j PVEFW-FWBR-OUT
exists PVEFW-FWBR-OUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore BROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore FORWARD (zuQi5YOvmMWiM9zohnQw/qWemOA)
ignore INPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore OUTPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore POSTROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore PREROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
no changes

Switching firewall back off

Connectivity is back now, I don't need to reboot the host

pve-firewall status

Code: 
root@kt-pve-01:~# pve-firewall status
Status: disabled/running

pve-firewall compile

Code: 
root@kt-pve-01:~# pve-firewall compile
ipset cmdlist:

iptables cmdlist:

ip6tables cmdlist:

ebtables cmdlist:
ignore BROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore FORWARD (zuQi5YOvmMWiM9zohnQw/qWemOA)
ignore INPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore OUTPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore POSTROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore PREROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
delete PVEFW-FORWARD (ULtZ6lqjrD/jAKLY+OZo3BbXs9k)
delete PVEFW-FWBR-OUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
no changes
firewall disabled

Turning off etables and firewall still off

pve-firewall status

Code: 
root@kt-pve-01:~# pve-firewall status
Status: disabled/running

pve-firewall compile

Code: 
root@kt-pve-01:~# pve-firewall compile
ipset cmdlist:

iptables cmdlist:

ip6tables cmdlist:

ebtables cmdlist:
ignore BROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore FORWARD (zuQi5YOvmMWiM9zohnQw/qWemOA)
ignore INPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore OUTPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore POSTROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore PREROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
delete PVEFW-FORWARD (ULtZ6lqjrD/jAKLY+OZo3BbXs9k)
delete PVEFW-FWBR-OUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
no changes
firewall disabled

Turning firewall on with etables off

Connectivity is broken again

pve-firewall status

Code: 
root@kt-pve-01:~# pve-firewall status
Status: enabled/running

pve-firewall compile

Code: 
root@kt-pve-01:~# pve-firewall compile
ipset cmdlist:
exists PVEFW-0-management-v4 (hwAfqIGn9k7XQSlehQ9eeyB5+uM)
        create PVEFW-0-management-v4 hash:net family inet hashsize 64 maxelem 64
        add PVEFW-0-management-v4 172.28.28.0/23
exists PVEFW-0-management-v6 (H5WO/Pkuyz4e7OLB2uiMpG0Bsn0)
        create PVEFW-0-management-v6 hash:net family inet6 hashsize 64 maxelem 64

iptables cmdlist:
exists PVEFW-Drop (WDy2wbFe7jNYEyoO3QhUELZ4mIQ)
        -A PVEFW-Drop -p tcp --dport 43 -j PVEFW-reject
        -A PVEFW-Drop  -j PVEFW-DropBroadcast
        -A PVEFW-Drop -p icmp -m icmp --icmp-type fragmentation-needed -j ACCEPT
        -A PVEFW-Drop -p icmp -m icmp --icmp-type time-exceeded -j ACCEPT
        -A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-Drop -p udp --match multiport --dports 135,445 -j DROP
        -A PVEFW-Drop -p udp --dport 137:139 -j DROP
        -A PVEFW-Drop -p udp --sport 137 --dport 1024:65535 -j DROP
        -A PVEFW-Drop -p tcp --match multiport --dports 135,139,445 -j DROP
        -A PVEFW-Drop -p udp --dport 1900 -j DROP
        -A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
        -A PVEFW-Drop -p udp --sport 53 -j DROP
exists PVEFW-DropBroadcast (NyjHNAtFbkH7WGLamPpdVnxHy4w)
        -A PVEFW-DropBroadcast -m addrtype --dst-type BROADCAST -j DROP
        -A PVEFW-DropBroadcast -m addrtype --dst-type MULTICAST -j DROP
        -A PVEFW-DropBroadcast -m addrtype --dst-type ANYCAST -j DROP
        -A PVEFW-DropBroadcast -d 224.0.0.0/4 -j DROP
exists PVEFW-FORWARD (qnNexOcGa+y+jebd4dAUqFSp5nw)
        -A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
        -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-in fwln+ -j PVEFW-FWBR-IN
        -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-out fwln+ -j PVEFW-FWBR-OUT
exists PVEFW-FWBR-IN (Ijl7/xz0DD7LF91MlLCz0ybZBE0)
        -A PVEFW-FWBR-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
exists PVEFW-FWBR-OUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-INPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-OUTPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-Reject (CZJnIN6rAdpu+ej59QPr9+laMUo)
        -A PVEFW-Reject -p tcp --dport 43 -j PVEFW-reject
        -A PVEFW-Reject  -j PVEFW-DropBroadcast
        -A PVEFW-Reject -p icmp -m icmp --icmp-type fragmentation-needed -j ACCEPT
        -A PVEFW-Reject -p icmp -m icmp --icmp-type time-exceeded -j ACCEPT
        -A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-Reject -p udp --match multiport --dports 135,445 -j PVEFW-reject
        -A PVEFW-Reject -p udp --dport 137:139 -j PVEFW-reject
        -A PVEFW-Reject -p udp --sport 137 --dport 1024:65535 -j PVEFW-reject
        -A PVEFW-Reject -p tcp --match multiport --dports 135,139,445 -j PVEFW-reject
        -A PVEFW-Reject -p udp --dport 1900 -j DROP
        -A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
        -A PVEFW-Reject -p udp --sport 53 -j DROP
exists PVEFW-SET-ACCEPT-MARK (Hg/OIgIwJChBUcWU8Xnjhdd2jUY)
        -A PVEFW-SET-ACCEPT-MARK  -j MARK --set-mark 0x80000000/0x80000000
exists PVEFW-logflags (MN4PH1oPZeABMuWr64RrygPfW7A)
        -A PVEFW-logflags  -j DROP
exists PVEFW-reject (Jlkrtle1mDdtxDeI9QaDSL++Npc)
        -A PVEFW-reject -m addrtype --dst-type BROADCAST -j DROP
        -A PVEFW-reject -s 224.0.0.0/4 -j DROP
        -A PVEFW-reject -p icmp -j DROP
        -A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
        -A PVEFW-reject -p udp -j REJECT --reject-with icmp-port-unreachable
        -A PVEFW-reject -p icmp -j REJECT --reject-with icmp-host-unreachable
        -A PVEFW-reject  -j REJECT --reject-with icmp-host-prohibited
exists PVEFW-smurflog (2gfT1VMkfr0JL6OccRXTGXo+1qk)
        -A PVEFW-smurflog  -j DROP
exists PVEFW-smurfs (HssVe5QCBXd5mc9kC88749+7fag)
        -A PVEFW-smurfs -s 0.0.0.0/32 -j RETURN
        -A PVEFW-smurfs -m addrtype --src-type BROADCAST -g PVEFW-smurflog
        -A PVEFW-smurfs -s 224.0.0.0/4 -g PVEFW-smurflog
exists PVEFW-tcpflags (CMFojwNPqllyqD67NeI5m+bP5mo)
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags

ip6tables cmdlist:
exists PVEFW-Drop (Jb79Uw7z1vZglIcV7QXA5uY/nbk)
        -A PVEFW-Drop -p tcp --dport 43 -j PVEFW-reject
        -A PVEFW-Drop  -j PVEFW-DropBroadcast
        -A PVEFW-Drop -p icmpv6 -m icmpv6 --icmpv6-type destination-unreachable -j ACCEPT
        -A PVEFW-Drop -p icmpv6 -m icmpv6 --icmpv6-type time-exceeded -j ACCEPT
        -A PVEFW-Drop -p icmpv6 -m icmpv6 --icmpv6-type packet-too-big -j ACCEPT
        -A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-Drop -p udp --match multiport --dports 135,445 -j DROP
        -A PVEFW-Drop -p udp --dport 137:139 -j DROP
        -A PVEFW-Drop -p udp --sport 137 --dport 1024:65535 -j DROP
        -A PVEFW-Drop -p tcp --match multiport --dports 135,139,445 -j DROP
        -A PVEFW-Drop -p udp --dport 1900 -j DROP
        -A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
        -A PVEFW-Drop -p udp --sport 53 -j DROP
exists PVEFW-DropBroadcast (8Krk5Nh8pDZOOc7BQAbM6PlyFSU)
        -A PVEFW-DropBroadcast -d ff00::/8 -j DROP
exists PVEFW-FORWARD (qnNexOcGa+y+jebd4dAUqFSp5nw)
        -A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
        -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-in fwln+ -j PVEFW-FWBR-IN
        -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-out fwln+ -j PVEFW-FWBR-OUT
exists PVEFW-FWBR-IN (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-FWBR-OUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-INPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-OUTPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
exists PVEFW-Reject (aL1nrxJk/u3XmTb3Am2eaM/3yCM)
        -A PVEFW-Reject -p tcp --dport 43 -j PVEFW-reject
        -A PVEFW-Reject  -j PVEFW-DropBroadcast
        -A PVEFW-Reject -p icmpv6 -m icmpv6 --icmpv6-type destination-unreachable -j ACCEPT
        -A PVEFW-Reject -p icmpv6 -m icmpv6 --icmpv6-type time-exceeded -j ACCEPT
        -A PVEFW-Reject -p icmpv6 -m icmpv6 --icmpv6-type packet-too-big -j ACCEPT
        -A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
        -A PVEFW-Reject -p udp --match multiport --dports 135,445 -j PVEFW-reject
        -A PVEFW-Reject -p udp --dport 137:139 -j PVEFW-reject
        -A PVEFW-Reject -p udp --sport 137 --dport 1024:65535 -j PVEFW-reject
        -A PVEFW-Reject -p tcp --match multiport --dports 135,139,445 -j PVEFW-reject
        -A PVEFW-Reject -p udp --dport 1900 -j DROP
        -A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
        -A PVEFW-Reject -p udp --sport 53 -j DROP
exists PVEFW-SET-ACCEPT-MARK (Hg/OIgIwJChBUcWU8Xnjhdd2jUY)
        -A PVEFW-SET-ACCEPT-MARK  -j MARK --set-mark 0x80000000/0x80000000
exists PVEFW-logflags (MN4PH1oPZeABMuWr64RrygPfW7A)
        -A PVEFW-logflags  -j DROP
exists PVEFW-reject (etEECUYcgUdzuuO+LDP83pu0S8Y)
        -A PVEFW-reject -p icmpv6 -j DROP
        -A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
        -A PVEFW-reject -p udp -j REJECT --reject-with icmp6-port-unreachable
        -A PVEFW-reject  -j REJECT --reject-with icmp6-adm-prohibited
exists PVEFW-tcpflags (CMFojwNPqllyqD67NeI5m+bP5mo)
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
        -A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags

ebtables cmdlist:
ignore BROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore FORWARD (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore INPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore OUTPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore POSTROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore PREROUTING (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
no changes

Proxmox version

Code: 
root@kt-pve-01:~# pveversion -v
proxmox-ve: 5.4-2 (running kernel: 4.15.18-16-pve)
pve-manager: 5.4-8 (running version: 5.4-8/51d494ca)
pve-kernel-4.15: 5.4-5
pve-kernel-4.15.18-17-pve: 4.15.18-43
pve-kernel-4.15.18-16-pve: 4.15.18-41
pve-kernel-4.15.18-12-pve: 4.15.18-36
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-11
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-53
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-13
libpve-storage-perl: 5.0-44
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-3
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-28
pve-cluster: 5.0-37
pve-container: 2.0-39
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-22
pve-firmware: 2.0-6
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 3.0.1-4
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-54
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.13-pve1~bpo2
 
okay, so it's not the missing ebtables cleanup ;) at a quick glance, I don't see anything in your pve-firewall output that should stop you from connecting to the guests. could you also include a sample guest config and output of "iptables-save" "ip6tables-save" and "ebtables-save" ?
 
Yes absolutely. Thanks for your help on this Fabian by the way.

VM 200 - Windows Server 2019 Core Domain Controller but it affects them all or at least I tested a few and all the same.

Code: 
agent: 1
balloon: 768
bootdisk: scsi0
cores: 1
cpu: host,flags=+pcid;+spec-ctrl
memory: 2048
name: kt-dc-01
net0: virtio=8A:B4:41:D2:CF:27,bridge=vmbr0
numa: 1
onboot: 1
ostype: win10
protection: 1
scsi0: ssd-m2:vm-200-disk-0,cache=writeback,discard=on,size=40G
scsi1: ssd-m2:vm-200-disk-1,discard=on,size=10G
scsihw: virtio-scsi-pci
smbios1: uuid=38876e2d-adda-4d6a-858f-bda9ec6a44aa
sockets: 1
startup: order=2
vmgenid: fe070f63-5c69-439a-b6f2-aaa43a45bd3a

iptables-save

Code: 
# Generated by iptables-save v1.6.0 on Wed Jul 10 23:50:37 2019
*security
:INPUT ACCEPT [63417154:58377399874]
:FORWARD ACCEPT [3406278559:2629238300206]
:OUTPUT ACCEPT [51548360:44696080782]
COMMIT
# Completed on Wed Jul 10 23:50:37 2019
# Generated by iptables-save v1.6.0 on Wed Jul 10 23:50:37 2019
*raw
:PREROUTING ACCEPT [3467895974:2692086278131]
:OUTPUT ACCEPT [51548360:44696080782]
COMMIT
# Completed on Wed Jul 10 23:50:37 2019
# Generated by iptables-save v1.6.0 on Wed Jul 10 23:50:37 2019
*nat
:PREROUTING ACCEPT [28130:9308553]
:INPUT ACCEPT [190:34504]
:OUTPUT ACCEPT [121:17287]
:POSTROUTING ACCEPT [25014:8017409]
COMMIT
# Completed on Wed Jul 10 23:50:37 2019
# Generated by iptables-save v1.6.0 on Wed Jul 10 23:50:37 2019
*mangle
:PREROUTING ACCEPT [3535252091:2776122070145]
:INPUT ACCEPT [67452223:64465428792]
:FORWARD ACCEPT [3469480418:2702149496213]
:OUTPUT ACCEPT [55551937:46639578789]
:POSTROUTING ACCEPT [3525038645:2748790090794]
COMMIT
# Completed on Wed Jul 10 23:50:37 2019
# Generated by iptables-save v1.6.0 on Wed Jul 10 23:50:37 2019
*filter
:INPUT ACCEPT [620:164271]
:FORWARD ACCEPT [15632:5800257]
:OUTPUT ACCEPT [609:201419]
:PVEFW-Drop - [0:0]
:PVEFW-DropBroadcast - [0:0]
:PVEFW-FORWARD - [0:0]
:PVEFW-FWBR-IN - [0:0]
:PVEFW-FWBR-OUT - [0:0]
:PVEFW-INPUT - [0:0]
:PVEFW-OUTPUT - [0:0]
:PVEFW-Reject - [0:0]
:PVEFW-SET-ACCEPT-MARK - [0:0]
:PVEFW-logflags - [0:0]
:PVEFW-reject - [0:0]
:PVEFW-smurflog - [0:0]
:PVEFW-smurfs - [0:0]
:PVEFW-tcpflags - [0:0]
-A INPUT -j PVEFW-INPUT
-A FORWARD -j PVEFW-FORWARD
-A OUTPUT -j PVEFW-OUTPUT
-A PVEFW-Drop -p tcp -m tcp --dport 43 -j PVEFW-reject
-A PVEFW-Drop -j PVEFW-DropBroadcast
-A PVEFW-Drop -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
-A PVEFW-Drop -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Drop -p udp -m multiport --dports 135,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 137:139 -j DROP
-A PVEFW-Drop -p udp -m udp --sport 137 --dport 1024:65535 -j DROP
-A PVEFW-Drop -p tcp -m multiport --dports 135,139,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Drop -p udp -m udp --sport 53 -j DROP
-A PVEFW-Drop -m comment --comment "PVESIG:WDy2wbFe7jNYEyoO3QhUELZ4mIQ"
-A PVEFW-DropBroadcast -m addrtype --dst-type BROADCAST -j DROP
-A PVEFW-DropBroadcast -m addrtype --dst-type MULTICAST -j DROP
-A PVEFW-DropBroadcast -m addrtype --dst-type ANYCAST -j DROP
-A PVEFW-DropBroadcast -d 224.0.0.0/4 -j DROP
-A PVEFW-DropBroadcast -m comment --comment "PVESIG:NyjHNAtFbkH7WGLamPpdVnxHy4w"
-A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
-A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-FORWARD -m physdev --physdev-in fwln+ --physdev-is-bridged -j PVEFW-FWBR-IN
-A PVEFW-FORWARD -m physdev --physdev-out fwln+ --physdev-is-bridged -j PVEFW-FWBR-OUT
-A PVEFW-FORWARD -m comment --comment "PVESIG:qnNexOcGa+y+jebd4dAUqFSp5nw"
-A PVEFW-FWBR-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A PVEFW-FWBR-IN -m comment --comment "PVESIG:Ijl7/xz0DD7LF91MlLCz0ybZBE0"
-A PVEFW-FWBR-OUT -m comment --comment "PVESIG:2jmj7l5rSw0yVb/vlWAYkK/YBwk"
-A PVEFW-INPUT -m comment --comment "PVESIG:2jmj7l5rSw0yVb/vlWAYkK/YBwk"
-A PVEFW-OUTPUT -m comment --comment "PVESIG:2jmj7l5rSw0yVb/vlWAYkK/YBwk"
-A PVEFW-Reject -p tcp -m tcp --dport 43 -j PVEFW-reject
-A PVEFW-Reject -j PVEFW-DropBroadcast
-A PVEFW-Reject -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
-A PVEFW-Reject -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Reject -p udp -m multiport --dports 135,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 137:139 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --sport 137 --dport 1024:65535 -j PVEFW-reject
-A PVEFW-Reject -p tcp -m multiport --dports 135,139,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Reject -p udp -m udp --sport 53 -j DROP
-A PVEFW-Reject -m comment --comment "PVESIG:CZJnIN6rAdpu+ej59QPr9+laMUo"
-A PVEFW-SET-ACCEPT-MARK -j MARK --set-xmark 0x80000000/0x80000000
-A PVEFW-SET-ACCEPT-MARK -m comment --comment "PVESIG:Hg/OIgIwJChBUcWU8Xnjhdd2jUY"
-A PVEFW-logflags -j DROP
-A PVEFW-logflags -m comment --comment "PVESIG:MN4PH1oPZeABMuWr64RrygPfW7A"
-A PVEFW-reject -m addrtype --dst-type BROADCAST -j DROP
-A PVEFW-reject -s 224.0.0.0/4 -j DROP
-A PVEFW-reject -p icmp -j DROP
-A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
-A PVEFW-reject -p udp -j REJECT --reject-with icmp-port-unreachable
-A PVEFW-reject -p icmp -j REJECT --reject-with icmp-host-unreachable
-A PVEFW-reject -j REJECT --reject-with icmp-host-prohibited
-A PVEFW-reject -m comment --comment "PVESIG:Jlkrtle1mDdtxDeI9QaDSL++Npc"
-A PVEFW-smurflog -j DROP
-A PVEFW-smurflog -m comment --comment "PVESIG:2gfT1VMkfr0JL6OccRXTGXo+1qk"
-A PVEFW-smurfs -s 0.0.0.0/32 -j RETURN
-A PVEFW-smurfs -m addrtype --src-type BROADCAST -g PVEFW-smurflog
-A PVEFW-smurfs -s 224.0.0.0/4 -g PVEFW-smurflog
-A PVEFW-smurfs -m comment --comment "PVESIG:HssVe5QCBXd5mc9kC88749+7fag"
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags
-A PVEFW-tcpflags -m comment --comment "PVESIG:CMFojwNPqllyqD67NeI5m+bP5mo"
COMMIT
# Completed on Wed Jul 10 23:50:37 2019

iptables6-save

Code: 
# Generated by ip6tables-save v1.6.0 on Wed Jul 10 23:47:08 2019
*nat
:PREROUTING ACCEPT [424:101028]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [293:47470]
COMMIT
# Completed on Wed Jul 10 23:47:08 2019
# Generated by ip6tables-save v1.6.0 on Wed Jul 10 23:47:08 2019
*mangle
:PREROUTING ACCEPT [248228751:70383440003]
:INPUT ACCEPT [88708:16433359]
:FORWARD ACCEPT [223048399:65956086571]
:OUTPUT ACCEPT [1622:128934]
:POSTROUTING ACCEPT [223049989:65956213513]
COMMIT
# Completed on Wed Jul 10 23:47:08 2019
# Generated by ip6tables-save v1.6.0 on Wed Jul 10 23:47:08 2019
*security
:INPUT ACCEPT [88708:16433359]
:FORWARD ACCEPT [223048367:65956084579]
:OUTPUT ACCEPT [1622:128934]
COMMIT
# Completed on Wed Jul 10 23:47:08 2019
# Generated by ip6tables-save v1.6.0 on Wed Jul 10 23:47:08 2019
*raw
:PREROUTING ACCEPT [248228751:70383440003]
:OUTPUT ACCEPT [1622:128934]
COMMIT
# Completed on Wed Jul 10 23:47:08 2019
# Generated by ip6tables-save v1.6.0 on Wed Jul 10 23:47:08 2019
*filter
:INPUT ACCEPT [34:5168]
:FORWARD ACCEPT [92805:16610249]
:OUTPUT ACCEPT [0:0]
:PVEFW-Drop - [0:0]
:PVEFW-DropBroadcast - [0:0]
:PVEFW-FORWARD - [0:0]
:PVEFW-FWBR-IN - [0:0]
:PVEFW-FWBR-OUT - [0:0]
:PVEFW-INPUT - [0:0]
:PVEFW-OUTPUT - [0:0]
:PVEFW-Reject - [0:0]
:PVEFW-SET-ACCEPT-MARK - [0:0]
:PVEFW-logflags - [0:0]
:PVEFW-reject - [0:0]
:PVEFW-tcpflags - [0:0]
-A INPUT -j PVEFW-INPUT
-A FORWARD -j PVEFW-FORWARD
-A OUTPUT -j PVEFW-OUTPUT
-A PVEFW-Drop -p tcp -m tcp --dport 43 -j PVEFW-reject
-A PVEFW-Drop -j PVEFW-DropBroadcast
-A PVEFW-Drop -p ipv6-icmp -m icmp6 --icmpv6-type 1 -j ACCEPT
-A PVEFW-Drop -p ipv6-icmp -m icmp6 --icmpv6-type 3 -j ACCEPT
-A PVEFW-Drop -p ipv6-icmp -m icmp6 --icmpv6-type 2 -j ACCEPT
-A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Drop -p udp -m multiport --dports 135,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 137:139 -j DROP
-A PVEFW-Drop -p udp -m udp --sport 137 --dport 1024:65535 -j DROP
-A PVEFW-Drop -p tcp -m multiport --dports 135,139,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Drop -p udp -m udp --sport 53 -j DROP
-A PVEFW-Drop -m comment --comment "PVESIG:Jb79Uw7z1vZglIcV7QXA5uY/nbk"
-A PVEFW-DropBroadcast -d ff00::/8 -j DROP
-A PVEFW-DropBroadcast -m comment --comment "PVESIG:8Krk5Nh8pDZOOc7BQAbM6PlyFSU"
-A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
-A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-FORWARD -m physdev --physdev-in fwln+ --physdev-is-bridged -j PVEFW-FWBR-IN
-A PVEFW-FORWARD -m physdev --physdev-out fwln+ --physdev-is-bridged -j PVEFW-FWBR-OUT
-A PVEFW-FORWARD -m comment --comment "PVESIG:qnNexOcGa+y+jebd4dAUqFSp5nw"
-A PVEFW-FWBR-IN -m comment --comment "PVESIG:2jmj7l5rSw0yVb/vlWAYkK/YBwk"
-A PVEFW-FWBR-OUT -m comment --comment "PVESIG:2jmj7l5rSw0yVb/vlWAYkK/YBwk"
-A PVEFW-INPUT -m comment --comment "PVESIG:2jmj7l5rSw0yVb/vlWAYkK/YBwk"
-A PVEFW-OUTPUT -m comment --comment "PVESIG:2jmj7l5rSw0yVb/vlWAYkK/YBwk"
-A PVEFW-Reject -p tcp -m tcp --dport 43 -j PVEFW-reject
-A PVEFW-Reject -j PVEFW-DropBroadcast
-A PVEFW-Reject -p ipv6-icmp -m icmp6 --icmpv6-type 1 -j ACCEPT
-A PVEFW-Reject -p ipv6-icmp -m icmp6 --icmpv6-type 3 -j ACCEPT
-A PVEFW-Reject -p ipv6-icmp -m icmp6 --icmpv6-type 2 -j ACCEPT
-A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Reject -p udp -m multiport --dports 135,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 137:139 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --sport 137 --dport 1024:65535 -j PVEFW-reject
-A PVEFW-Reject -p tcp -m multiport --dports 135,139,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Reject -p udp -m udp --sport 53 -j DROP
-A PVEFW-Reject -m comment --comment "PVESIG:aL1nrxJk/u3XmTb3Am2eaM/3yCM"
-A PVEFW-SET-ACCEPT-MARK -j MARK --set-xmark 0x80000000/0x80000000
-A PVEFW-SET-ACCEPT-MARK -m comment --comment "PVESIG:Hg/OIgIwJChBUcWU8Xnjhdd2jUY"
-A PVEFW-logflags -j DROP
-A PVEFW-logflags -m comment --comment "PVESIG:MN4PH1oPZeABMuWr64RrygPfW7A"
-A PVEFW-reject -p ipv6-icmp -j DROP
-A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
-A PVEFW-reject -p udp -j REJECT --reject-with icmp6-port-unreachable
-A PVEFW-reject -j REJECT --reject-with icmp6-adm-prohibited
-A PVEFW-reject -m comment --comment "PVESIG:etEECUYcgUdzuuO+LDP83pu0S8Y"
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags
-A PVEFW-tcpflags -m comment --comment "PVESIG:CMFojwNPqllyqD67NeI5m+bP5mo"
COMMIT
# Completed on Wed Jul 10 23:47:08 2019

ebtables-save

Code: 
# Generated by ebtables-save v1.0 on Wed Jul 10 23:47:44 BST 2019
*nat
:PREROUTING ACCEPT
:OUTPUT ACCEPT
:POSTROUTING ACCEPT

*broute
:BROUTING ACCEPT

*filter
:INPUT ACCEPT
:FORWARD ACCEPT
:OUTPUT ACCEPT
:BROUTING ACCEPT
:POSTROUTING ACCEPT
:PREROUTING ACCEPT
:PVEFW-FORWARD ACCEPT
:PVEFW-FWBR-OUT ACCEPT
-A FORWARD -j PVEFW-FORWARD
-A PVEFW-FORWARD -p IPv4 -j ACCEPT
-A PVEFW-FORWARD -p IPv6 -j ACCEPT
-A PVEFW-FORWARD -o fwln+ -j PVEFW-FWBR-OUT

And just if it helps my interfaces.

Code: 
 network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual
#1Gbit NIC 01

iface eno2 inet manual
#1Gbit NIC 02

iface eno3 inet manual
#10Gbit NIC 01

iface eno4 inet manual
#10Gbit NIC 02

auto vmbr0
iface vmbr0 inet static
    address  172.28.28.5
    netmask  23
    gateway  172.28.28.1
    bridge-ports eno3
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
#LAN

auto vmbr9
iface vmbr9 inet manual
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
#Host Only

I've not given the server an IPv6 address but I do have a dual stack WAN link and clients behind this get IPv6 router advertisements such that they can pass an IPv6 site test. Just haven't felt the need to dual stack my servers.
 
that output looks okay/standard. I guess the next step is to manually debug where the packets get lost (e.g., via tcpdump along the way/on each interface).
 
H Fabian,

Sorry for the delay. I’ve had another look into this and it’s not across the board. I’ve noticed that it seems when crossing a VLAN it stays working. So an example.

Laptop -> VM on same VLAN = No connectivity

Laptop -> OPNsense Firewall (Virtual on Proxmox but with 2 x PCI passthrough NIC’s) -> Out of one of those NIC’s and back in another now on a different VLAN -> VM on different VLAN = This one works with the Proxmox firewall on

I’ve also installed a Proxmox 6 beta 1 at the time into a nested VM and tried to test as many configurations as I can and the firewall works perfectly all the time in that instance).

Hence I’m going to backup now 6 final is out and rebuild the server from scratch plus keep testing the firewall. See if I can happen across the combination that causes this.

I do normally have one VM with a passthrough tun for OpenVPN and have since started running nested Docker in LXC since 5.3 came out. Though not your fault but there’s issues with that on storage drivers plus the odd Docker like Collebera that wants mknod which doesn’t work. So I’ll probably go to a VM on rebuild and give virtio 9p a whirl as a way to have fast access to storage still.

Anyway will let you know how I get on in a bit.
 
OK so installed PVE 6 and switched the firewall on straight away. All seems good on a clean install.

Then restored a few VM's and Containers and then it's broken as it was on PVE 5. I believe it's one of the following that might be causing it but not sure which. Will try a reboot later after stopping all VM's on start to see if it works after a reboot then start them one by one. Will the firewall tables all be cleared on a reboot or are they saved so they persist?

The configuration that might be doing would be:

Privileged LXC running OpenVPN

lxc.cgroup.devices.allow: c 10:200 rwm
lxc.hook.autodev: sh -c "modprobe tun; cd ${LXC_ROOTFS_MOUNT}/dev; mkdir net; mknod net/tun c 10 200; chmod 0666 net/tun"

Suspect this is the most likely

Docker inside an unprivileged LXC

features: keyctl=1,nesting=1
 
OK rebooted the server, no VM's or containers starting on bootup. Switching on a single QEMU based VM running Windows. Ping it and all good. Turn on the firewall on the Datacentre and there's no firewall enabled on this VM and ping stops working. Switch Datacentre back off and it works again.

Removing rules one by one from IPTables and it all starts working again when I remove the following rule. Just to be sure if I switch the firewall off and on so they all recreate, this is the only one I have to remove for it all to start working again.

Chain PVEFW-FORWARD (1 references)
num target prot opt source destination
1 DROP all -- anywhere anywhere ctstate INVALID

I've turned off all NIC offloading but that doesn't help either but at least the checksums show correct in tcpdump...

tcpdump with Firewall off

Testing port 3389 from laptop to VM

Code: 
tcpdump -vv -i eno3 dst 172.28.28.50
tcpdump: listening on eno3, link-type EN10MB (Ethernet), capture size 262144 bytes
20:04:20.469573 IP (tos 0x0, ttl 128, id 31079, offset 0, flags [DF], proto TCP (6), length 52)
    KT-PC-01.domain.com.3941 > KT-RD-01.domain.com.3389: Flags [S], cksum 0x260c (correct), seq 1025467008, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
20:04:20.470432 IP (tos 0x0, ttl 128, id 31080, offset 0, flags [DF], proto TCP (6), length 40)
    KT-PC-01.domain.com.3941 > KT-RD-01.domain.com.3389: Flags [.], cksum 0xee6f (correct), seq 1025467009, ack 2772479483, win 8212, length 0
20:04:20.470650 IP (tos 0x0, ttl 128, id 31081, offset 0, flags [DF], proto TCP (6), length 40)
    KT-PC-01.domain.com.3941 > KT-RD-01.domain.com.3389: Flags [F.], cksum 0xee6e (correct), seq 0, ack 1, win 8212, length 0
20:04:20.470963 IP (tos 0x0, ttl 128, id 31082, offset 0, flags [DF], proto TCP (6), length 40)
    KT-PC-01.domain.com.3919 > KT-RD-01.domain.com.3389: Flags [.], cksum 0x7dc8 (correct), seq 3488519609, ack 1145878949, win 8212, length 0
20:04:20.473779 IP (tos 0x0, ttl 128, id 31083, offset 0, flags [none], proto UDP (17), length 78)
    KT-PC-01.domain.com.netbios-ns > KT-RD-01.domain.com.netbios-ns: [udp sum ok]
>>> NBT UDP PACKET(137): QUERY; REQUEST; UNICAST
TrnID=0xB3C2
OpCode=0
NmFlags=0x0
Rcode=0
QueryCount=1
AnswerCount=0
AuthorityCount=0
AddressRecCount=0
QuestionRecords:
Name=*               NameType=0x00 (Workstation)
QuestionType=0x21
QuestionClass=0x1


Same with Firewall on a straight after

Code: 
tcpdump -vv -i vmbr0 dst 172.28.28.50
tcpdump: listening on vmbr0, link-type EN10MB (Ethernet), capture size 262144 bytes
20:05:24.031679 IP (tos 0x0, ttl 128, id 31084, offset 0, flags [DF], proto TCP (6), length 52)
    KT-PC-01.domain.com.3958 > KT-RD-01.domain.com.3389: Flags [S], cksum 0xac66 (correct), seq 1043389187, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
20:05:25.031812 IP (tos 0x0, ttl 128, id 31085, offset 0, flags [DF], proto TCP (6), length 52)
    KT-PC-01.domain.com.3958 > KT-RD-01.domain.com.3389: Flags [S], cksum 0xac66 (correct), seq 1043389187, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
20:05:27.032811 IP (tos 0x0, ttl 128, id 31086, offset 0, flags [DF], proto TCP (6), length 52)
    KT-PC-01.domain.com.3958 > KT-RD-01.domain.com.3389: Flags [S], cksum 0xac66 (correct), seq 1043389187, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
20:05:28.637175 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has KT-RD-01.domain.com (4e:2d:19:05:1a:5c (oui Unknown)) tell KT-PC-01.domain.com, length 46
20:05:28.941229 ARP, Ethernet (len 6), IPv4 (len 4), Reply KT-PC-01.domain.com is-at 30:9c:23:50:3a:0c (oui Unknown), length 46
20:05:31.032656 IP (tos 0x0, ttl 128, id 31087, offset 0, flags [DF], proto TCP (6), length 52)
    KT-PC-01.domain.com.3958 > KT-RD-01.domain.com.3389: Flags [S], cksum 0xac66 (correct), seq 1043389187, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
20:05:39.032486 IP (tos 0x0, ttl 128, id 31088, offset 0, flags [DF], proto TCP (6), length 52)
    KT-PC-01.domain.com.3958 > KT-RD-01.domain.com.3389: Flags [S], cksum 0xac66 (correct), seq 1043389187, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
20:05:45.075373 IP (tos 0x0, ttl 128, id 31089, offset 0, flags [none], proto ICMP (1), length 60)
    KT-PC-01.domain.com > KT-RD-01.domain.com: ICMP echo request, id 1, seq 703, length 40
20:05:49.673340 IP (tos 0x0, ttl 128, id 31090, offset 0, flags [none], proto UDP (17), length 78)
    KT-PC-01.domain.com.netbios-ns > KT-RD-01.domain.com.netbios-ns: [udp sum ok]
>>> NBT UDP PACKET(137): QUERY; REQUEST; UNICAST
TrnID=0xB3CC
OpCode=0
NmFlags=0x0
Rcode=0
QueryCount=1
AnswerCount=0
AuthorityCount=0
AddressRecCount=0
QuestionRecords:
Name=*               NameType=0x00 (Workstation)
QuestionType=0x21
QuestionClass=0x1


Hope this helps. Not sure where else to go from here.
 
I think I've solved it but only by working around it and switching to OpenVSwitch from Linux Bridges. It seems to work OK with OpenVSwitch.
 
FastLaneJB said:
OK rebooted the server, no VM's or containers starting on bootup. Switching on a single QEMU based VM running Windows. Ping it and all good. Turn on the firewall on the Datacentre and there's no firewall enabled on this VM and ping stops working. Switch Datacentre back off and it works again.

Removing rules one by one from IPTables and it all starts working again when I remove the following rule. Just to be sure if I switch the firewall off and on so they all recreate, this is the only one I have to remove for it all to start working again.

Chain PVEFW-FORWARD (1 references)
num target prot opt source destination
1 DROP all -- anywhere anywhere ctstate INVALID
Click to expand...

I did the very same and it works here. seems like something in your network interferes with connection tracking somehow? you can set "nf_conntrack_allow_invalid" in the host firewall config, which will cause exactly that rule to not be generated ;) log_nf_conntrack might also be interesting in case you want to debug this further..
 
fabian said:
I did the very same and it works here. seems like something in your network interferes with connection tracking somehow? you can set "nf_conntrack_allow_invalid" in the host firewall config, which will cause exactly that rule to not be generated ;) log_nf_conntrack might also be interesting in case you want to debug this further..
Click to expand...

Yeah I don't get it, I fired up an old server which has 5.3 on it in the same network, hell I even tried putting it on the same cable in the same port and it works fine. So I honestly don't understand what the issue is with the other server but seems Openvswitch gets around the issue.

I'll maybe look into the log_nf_conntrack in a bit as still curious what the issue really is.
 
Hi,

My problem persists, too. But in my case, If I set the firewall off after being on I don't recover connectivity. I have to reboot the node. Well, reboot the host. Because reboot from the panel doesn't work too.

My iptables-save after setting firewall off:

Code: 
# Generated by iptables-save v1.6.0 on Wed Jul 24 01:59:07 2019
*filter
:INPUT ACCEPT [8820:2031912]
:FORWARD ACCEPT [3230:247988]
:OUTPUT ACCEPT [6700:2451482]
COMMIT
# Completed on Wed Jul 24 01:59:07 2019
# Generated by iptables-save v1.6.0 on Wed Jul 24 01:59:07 2019
*nat
:PREROUTING ACCEPT [1896:202634]
:INPUT ACCEPT [850:51466]
:OUTPUT ACCEPT [431:29066]
:POSTROUTING ACCEPT [594:39529]
-A POSTROUTING -s 10.0.0.0/24 -o vmbr0 -j MASQUERADE
COMMIT
# Completed on Wed Jul 24 01:59:07 2019

My ebtables-save after setting firewall off:

Code: 
# Generated by ebtables-save v1.0 on Wed Jul 24 01:59:53 CEST 2019
*filter
:INPUT ACCEPT
:FORWARD ACCEPT
:OUTPUT ACCEPT

My pveversion -v

Code: 
proxmox-ve: 5.4-2 (running kernel: 4.15.18-18-pve)
pve-manager: 5.4-11 (running version: 5.4-11/6df3d8d0)
pve-kernel-4.15: 5.4-6
pve-kernel-4.15.18-18-pve: 4.15.18-44
pve-kernel-4.15.18-17-pve: 4.15.18-43
pve-kernel-4.15.18-15-pve: 4.15.18-40
pve-kernel-4.15.18-14-pve: 4.15.18-39
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-12
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-53
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-14
libpve-storage-perl: 5.0-44
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-3
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-28
pve-cluster: 5.0-37
pve-container: 2.0-39
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-22
pve-firmware: 2.0-6
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 3.0.1-4
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-54
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3

My pve-firewall status:

Code: 
Status: disabled/running

My pve-firewall compile:

Code: 
ipset cmdlist:

iptables cmdlist:

ip6tables cmdlist:

ebtables cmdlist:
ignore FORWARD (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore INPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
ignore OUTPUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
no changes
firewall disabled
 
I actually just hit this exact issue today as well, on a cluster of 3 servers hosted in Hetzner, accessed/managed via a Hetzner vSwitch. Enabling the firewall (even with ACCEPT/ACCEPT at the Datacenter level) resulted in losing connectivity to all VMs and management via other (non-local) subnets.

I added "nf_conntrack_allow_invalid: 1" to each of my node firewall configs, restarted with "pve-firewall restart", and the problem went away and the firewall appears to now operate normally as I would expect.

I can't see any reason for this to be an issue, but maybe there is something strange going on here. If I can help with any additional debug or troubleshooting, let me know - my cluster is not in production yet so happy to test/break/play if it helps :)

Thanks!

Rob.
 
Thanks wasteground.

I've tried your workaround but unfortunately, it doesn't work in my case. I have just one node and after applying the option in the pve-firewall and restarting it, nothing changes :(.

Any idea?
 
Hi Rob,

Sorry to bother you. Any idea about how could I troublshoot my problem? The main problem is that I cannot understand why this is not working. If I look at iptables all seems to be ok, but definitely doesn't work.

Best regards.
 
Apologies for the slow reply here. In my case, originally I was installing Proxmox on top of Debian 10, and the fix was just to add the "nf_conntrack_allow_invalid: 1" in the host.fw for each node - I didn't have to do anything other than that.

What I did then do is re-install all my Proxmox nodes using the Proxmox ISO, and I noticed I no longer needed the host.fw config at all - everything worked just as it should without needing this 'fix'

I'm not sure if that helps or not, but maybe it at least provides some ideas on how to proceed, but since I am no longer seeing the same issue I did not spend any additional time troubleshooting this. What I did learn is to basically just keep using the Proxmox ISO, rather than installing on top of an existing Debian installation :)
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back