Firewall one way connection problem

Discussion in 'Proxmox VE: Networking and Firewall' started by Eris, Jul 27, 2018.

  1. Eris

    Eris New Member

    Joined:
    Dec 16, 2015
    Messages:
    8
    Likes Received:
    0
    Hi,

    I have a connection problem with public IPs between 2 vms.

    First whats happening:

    I have 2 VMs each with 2 network interfaces all bound to the same vmbr on the same host.
    Firewall is disabled, at least on the node and on both vms. No other VM is on this node.

    ipfilter-netX is renamed for both VMs
    All Filters that are not disabled by the firewall are disabled (why the hell is a filter active if firewall is disabled?)

    If ip ping from VM1001 to VM5102 I get a ICMP reply and it also travels the correct interfaces.

    Now if I try to make a tcp request, for example wget I doesn't get a reply. The Syn-ack never comes back but it got sent by the VM5102.

    Debugging starts, I followed the flow...

    If I understand the flow correct it should flow the following way:

    Syn:
    VM1001-eth1
    node31-tap1001i1
    node31-fwbr1001i1
    node31-fwln1001i1
    node31-fwpr1001p1
    node31-vmbr0
    node31-fwpr5102p1
    node31-fwln5102i1
    node31-fwbr5102i1
    node31-tap5102i1
    VM5102-eth1

    Syn-ack:
    VM5102-eth1
    node31-tap5102i1
    node31-fwbr5102i1
    node31-fwln5102i1
    node31-fwpr5102p1
    node31-vmbr0 - last time I see the package in tcpdump
    node31-fwpr1001p1
    node31-fwln1001i1
    node31-fwbr1001i1
    node31-tap1001i1
    VM1001-eth1


    Even if I add ACCEPT to all iptables Rules it doesn't work
    # iptables -I INPUT -j ACCEPT
    # iptables -I FORWARD -j ACCEPT
    # iptables -I OUTPUT -j ACCEPT


    It doesn't matter if I move one of the VMs to another node its the same but if I move the IP to another cluster it works without problems...


    I tried the same IP von eth0 on both VMs but with no luck. If I use an internal 172.* I have no connection problems.

    VMs are Debian, Node got updated today.

    I would take any help...

    --- IPtables with noting in it....
    # Generated by iptables-save v1.6.0 on Fri Jul 27 19:17:54 2018
    *filter
    :INPUT ACCEPT [0:0]
    :FORWARD ACCEPT [0:0]
    :OUTPUT ACCEPT [1:180]
    :pVEFW-Drop - [0:0]
    :pVEFW-DropBroadcast - [0:0]
    :pVEFW-FORWARD - [0:0]
    :pVEFW-FWBR-IN - [0:0]
    :pVEFW-FWBR-OUT - [0:0]
    :pVEFW-INPUT - [0:0]
    :pVEFW-OUTPUT - [0:0]
    :pVEFW-Reject - [0:0]
    :pVEFW-SET-ACCEPT-MARK - [0:0]
    :pVEFW-logflags - [0:0]
    :pVEFW-reject - [0:0]
    :pVEFW-smurflog - [0:0]
    :pVEFW-smurfs - [0:0]
    :pVEFW-tcpflags - [0:0]
    :tap1001i0-IN - [0:0]
    :tap1001i0-OUT - [0:0]
    :tap1001i1-IN - [0:0]
    :tap1001i1-OUT - [0:0]
    :tap5102i0-IN - [0:0]
    :tap5102i0-OUT - [0:0]
    :tap5102i1-IN - [0:0]
    :tap5102i1-OUT - [0:0]
    -A INPUT -j ACCEPT
    -A INPUT -j PVEFW-INPUT
    -A FORWARD -j ACCEPT
    -A FORWARD -j PVEFW-FORWARD
    -A OUTPUT -j ACCEPT
    -A OUTPUT -j PVEFW-OUTPUT
    -A PVEFW-Drop -p tcp -m tcp --dport 43 -j PVEFW-reject
    -A PVEFW-Drop -j PVEFW-DropBroadcast
    -A PVEFW-Drop -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
    -A PVEFW-Drop -p icmp -m icmp --icmp-type 11 -j ACCEPT
    -A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
    -A PVEFW-Drop -p udp -m multiport --dports 135,445 -j DROP
    -A PVEFW-Drop -p udp -m udp --dport 137:139 -j DROP
    -A PVEFW-Drop -p udp -m udp --sport 137 --dport 1024:65535 -j DROP
    -A PVEFW-Drop -p tcp -m multiport --dports 135,139,445 -j DROP
    -A PVEFW-Drop -p udp -m udp --dport 1900 -j DROP
    -A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
    -A PVEFW-Drop -p udp -m udp --sport 53 -j DROP
    -A PVEFW-Drop -m comment --comment "PVESIG:WDy2wbFe7jNYEyoO3QhUELZ4mIQ"
    -A PVEFW-DropBroadcast -m addrtype --dst-type BROADCAST -j DROP
    -A PVEFW-DropBroadcast -m addrtype --dst-type MULTICAST -j DROP
    -A PVEFW-DropBroadcast -m addrtype --dst-type ANYCAST -j DROP
    -A PVEFW-DropBroadcast -d 224.0.0.0/4 -j DROP
    -A PVEFW-DropBroadcast -m comment --comment "PVESIG:NyjHNAtFbkH7WGLamPpdVnxHy4w"
    -A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
    -A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
    -A PVEFW-FORWARD -m physdev --physdev-in fwln+ --physdev-is-bridged -j PVEFW-FWBR-IN
    -A PVEFW-FORWARD -m physdev --physdev-out fwln+ --physdev-is-bridged -j PVEFW-FWBR-OUT
    -A PVEFW-FORWARD -m comment --comment "PVESIG:qnNexOcGa+y+jebd4dAUqFSp5nw"
    -A PVEFW-FWBR-IN -m physdev --physdev-out tap1001i0 --physdev-is-bridged -j tap1001i0-IN
    -A PVEFW-FWBR-IN -m physdev --physdev-out tap1001i1 --physdev-is-bridged -j tap1001i1-IN
    -A PVEFW-FWBR-IN -m physdev --physdev-out tap5102i0 --physdev-is-bridged -j tap5102i0-IN
    -A PVEFW-FWBR-IN -m physdev --physdev-out tap5102i1 --physdev-is-bridged -j tap5102i1-IN
    -A PVEFW-FWBR-IN -m comment --comment "PVESIG:xX/AgNcBbLheTc5eLDeU2DPBBD4"
    -A PVEFW-FWBR-OUT -m physdev --physdev-in tap1001i0 --physdev-is-bridged -j tap1001i0-OUT
    -A PVEFW-FWBR-OUT -m physdev --physdev-in tap1001i1 --physdev-is-bridged -j tap1001i1-OUT
    -A PVEFW-FWBR-OUT -m physdev --physdev-in tap5102i0 --physdev-is-bridged -j tap5102i0-OUT
    -A PVEFW-FWBR-OUT -m physdev --physdev-in tap5102i1 --physdev-is-bridged -j tap5102i1-OUT
    -A PVEFW-FWBR-OUT -m comment --comment "PVESIG:KyOynIUH0H9dk2Al5lV0Lmshp8s"
    -A PVEFW-INPUT -m comment --comment "PVESIG:2jmj7l5rSw0yVb/vlWAYkK/YBwk"
    -A PVEFW-OUTPUT -m comment --comment "PVESIG:2jmj7l5rSw0yVb/vlWAYkK/YBwk"
    -A PVEFW-Reject -p tcp -m tcp --dport 43 -j PVEFW-reject
    -A PVEFW-Reject -j PVEFW-DropBroadcast
    -A PVEFW-Reject -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
    -A PVEFW-Reject -p icmp -m icmp --icmp-type 11 -j ACCEPT
    -A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
    -A PVEFW-Reject -p udp -m multiport --dports 135,445 -j PVEFW-reject
    -A PVEFW-Reject -p udp -m udp --dport 137:139 -j PVEFW-reject
    -A PVEFW-Reject -p udp -m udp --sport 137 --dport 1024:65535 -j PVEFW-reject
    -A PVEFW-Reject -p tcp -m multiport --dports 135,139,445 -j PVEFW-reject
    -A PVEFW-Reject -p udp -m udp --dport 1900 -j DROP
    -A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
    -A PVEFW-Reject -p udp -m udp --sport 53 -j DROP
    -A PVEFW-Reject -m comment --comment "PVESIG:CZJnIN6rAdpu+ej59QPr9+laMUo"
    -A PVEFW-SET-ACCEPT-MARK -j MARK --set-xmark 0x80000000/0x80000000
    -A PVEFW-SET-ACCEPT-MARK -m comment --comment "PVESIG:Hg/OIgIwJChBUcWU8Xnjhdd2jUY"
    -A PVEFW-logflags -j NFLOG --nflog-prefix ":0:7:pVEFW-logflags: DROP: "
    -A PVEFW-logflags -j DROP
    -A PVEFW-logflags -m comment --comment "PVESIG:xxngynQ61gj3oDwdvenmOrWc1Z4"
    -A PVEFW-reject -m addrtype --dst-type BROADCAST -j DROP
    -A PVEFW-reject -s 224.0.0.0/4 -j DROP
    -A PVEFW-reject -p icmp -j DROP
    -A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
    -A PVEFW-reject -p udp -j REJECT --reject-with icmp-port-unreachable
    -A PVEFW-reject -p icmp -j REJECT --reject-with icmp-host-unreachable
    -A PVEFW-reject -j REJECT --reject-with icmp-host-prohibited
    -A PVEFW-reject -m comment --comment "PVESIG:Jlkrtle1mDdtxDeI9QaDSL++Npc"
    -A PVEFW-smurflog -j NFLOG --nflog-prefix ":0:7:pVEFW-smurflog: DROP: "
    -A PVEFW-smurflog -j DROP
    -A PVEFW-smurflog -m comment --comment "PVESIG:07iN6Ltw+eq1SF8lRxwoE+285nY"
    -A PVEFW-smurfs -s 0.0.0.0/32 -j RETURN
    -A PVEFW-smurfs -m addrtype --src-type BROADCAST -g PVEFW-smurflog
    -A PVEFW-smurfs -s 224.0.0.0/4 -g PVEFW-smurflog
    -A PVEFW-smurfs -m comment --comment "PVESIG:HssVe5QCBXd5mc9kC88749+7fag"
    -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
    -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
    -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
    -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
    -A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags
    -A PVEFW-tcpflags -m comment --comment "PVESIG:CMFojwNPqllyqD67NeI5m+bP5mo"
    -A tap1001i0-IN -j ACCEPT
    -A tap1001i0-IN -m comment --comment "PVESIG:6/i7HBviTdWE95tNd8maDixkN3E"
    -A tap1001i0-OUT -j MARK --set-xmark 0x0/0x80000000
    -A tap1001i0-OUT -g PVEFW-SET-ACCEPT-MARK
    -A tap1001i0-OUT -m comment --comment "PVESIG:uemS0BGWAeqPZ3b3A4NgT9Wbbmw"
    -A tap1001i1-IN -j ACCEPT
    -A tap1001i1-IN -m comment --comment "PVESIG:qRCoc51l0vdkcryTXojKu9AK3/A"
    -A tap1001i1-OUT -j MARK --set-xmark 0x0/0x80000000
    -A tap1001i1-OUT -g PVEFW-SET-ACCEPT-MARK
    -A tap1001i1-OUT -m comment --comment "PVESIG:0Y4PTnzdNh/JlSxU7/2ZL+OG0OU"
    -A tap5102i0-IN -j ACCEPT
    -A tap5102i0-IN -m comment --comment "PVESIG:E8T2Tx9IZCEDxqWyulmH6E4Ksl8"
    -A tap5102i0-OUT -j MARK --set-xmark 0x0/0x80000000
    -A tap5102i0-OUT -g PVEFW-SET-ACCEPT-MARK
    -A tap5102i0-OUT -m comment --comment "PVESIG:j/ITcVK+fqo20H7rn5AVoJoC+IU"
    -A tap5102i1-IN -j ACCEPT
    -A tap5102i1-IN -m comment --comment "PVESIG:Y8RDkaJFL3JZcPSZ/273Keu2Vac"
    -A tap5102i1-OUT -j MARK --set-xmark 0x0/0x80000000
    -A tap5102i1-OUT -g PVEFW-SET-ACCEPT-MARK
    -A tap5102i1-OUT -m comment --comment "PVESIG:hX3uhF/mmg9ndsWXFnbHyRctWKo"
    COMMIT
    # Completed on Fri Jul 27 19:17:54 2018

    ----



    thx

    Harald
     
  2. Richard

    Richard Proxmox Staff Member
    Staff Member

    Joined:
    Mar 6, 2015
    Messages:
    390
    Likes Received:
    10
    Sound contradictory - in order to clarify the situation: post the pve firewall-setting, the easiest way to get is

    Code:
    cd /etc/pve
    grep -r "" *.fw
    
    I guess that's rather a routing problem than a firewall one. Verify routing tables and IP settings in VMs and host by

    Code:
    ip addr
    ip route
    
     
  3. Eris

    Eris New Member

    Joined:
    Dec 16, 2015
    Messages:
    8
    Likes Received:
    0
    I will sent the .fw configs later, but I'm not sure how it could be a routing problem in an bridged Environment.

    Both VMs have the same subnet and the host doesn't have this subnet configured.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice