[SOLVED] Cluster Firewall Not Applying

Discussion in 'Proxmox VE: Networking and Firewall' started by HE_Cole, Dec 1, 2018.

  1. HE_Cole

    HE_Cole Member
    Proxmox VE Subscriber

    Joined:
    Oct 25, 2018
    Messages:
    34
    Likes Received:
    0
    Hey Everyone!

    I am sure this is something simple i missed but i setup all my firewall rules at the data center level in PVE and yet none of them are applied.

    First off.

    The firewall is enabled in Datacenter - Firewall Options.
    the firewall is enabled on each Node.
    I only set rules at the Datacenter level not at the node level. "thought that was ok"
    For my corosync interface node to node i allow all in/out from corosync ips any port tcp/udp.
    For wan ip node to node i allow all in/out any port tcp/udp.
    For ceph ip node to node i allow all in/out any port tcp/udp.
    just for safety on those interfaces.

    My data center input policy is DROP
    My data center output policy is ALLOW

    Question? Do i need a explict deny any any rule in my rules to block traffic that does not match a rule?

    And........

    If i run
    pve-firewall status
    Status: enabled/running (pending changes)

    I have tried restarting and stopping but no go rules are not applied.

    As a note too my management interface vmbr0 is also my wan interface for vms and vms and management are on the same subnet but i do have specific rules for the pve host/node ips.

    PVE complie

    Code:
    ipset cmdlist:
    exists PVEFW-0-management-v4 (LRAv4c0S4qV/TUNkSvLtGKJQpJE)
            create PVEFW-0-management-v4 hash:net family inet hashsize 64 maxelem 64
            add PVEFW-0-management-v4 23.136.0.0/24
    exists PVEFW-0-management-v6 (H5WO/Pkuyz4e7OLB2uiMpG0Bsn0)
            create PVEFW-0-management-v6 hash:net family inet6 hashsize 64 maxelem 64
    exists PVEFW-0-ssh-allow-v4 (n9JH7aMRhgUCAGOwXL7RkupiBZE)
            create PVEFW-0-ssh-allow-v4 hash:net family inet hashsize 64 maxelem 64
            add PVEFW-0-ssh-allow-v4 104.15.112.69
    exists PVEFW-0-ssh-allow-v6 (ct5hHx8Gukgtr7THZJRCjrVLof4)
            create PVEFW-0-ssh-allow-v6 hash:net family inet6 hashsize 64 maxelem 64
    update PVEFW-0-web-access-v4 (pbf6asNDW1MayeJDxWc8hZX9LHg)
            create PVEFW-0-web-access-v4 hash:net family inet hashsize 64 maxelem 64
            add PVEFW-0-web-access-v4 0.0.0.0/0
    update PVEFW-0-web-access-v6 (ZPI09T8PUwHkYSFBjsrF/ONEEN0)
            create PVEFW-0-web-access-v6 hash:net family inet6 hashsize 64 maxelem 64
            add PVEFW-0-web-access-v6 ::/0
    exists PVEFW-20261E6F (kzujSNsHcczuZLljf+bsAZBdP/4)
            create PVEFW-20261E6F hash:net family inet6 hashsize 64 maxelem 64
            add PVEFW-20261E6F 2606:8d80::/32
    exists PVEFW-22262195 (sq4e++1qTpAVxQrX834bxJlDlNU)
            create PVEFW-22262195 hash:net family inet hashsize 64 maxelem 64
            add PVEFW-22262195 23.136.0.1
            add PVEFW-22262195 23.136.0.3
            add PVEFW-22262195 23.136.0.4
            add PVEFW-22262195 23.136.0.5
            add PVEFW-22262195 23.136.0.6
            add PVEFW-22262195 23.136.0.7
    exists PVEFW-45430013 (/cthb6GW/B9wfOEfmHVXtEJdWKc)
            create PVEFW-45430013 hash:net family inet hashsize 64 maxelem 64
            add PVEFW-45430013 10.10.11.0/24
            add PVEFW-45430013 10.10.12.0/24
    exists PVEFW-47430339 (TdBHOcfofBIOCkWg17xmWS1uI+I)
            create PVEFW-47430339 hash:net family inet6 hashsize 64 maxelem 64
    exists PVEFW-5949F7C3 (ds34PpEWCcirgpWXBXsFfrWxTGI)
            create PVEFW-5949F7C3 hash:net family inet6 hashsize 64 maxelem 64
    exists PVEFW-5B49FAE9 (WpHu1/y6R4CXxYmIdxwdG55tOQY)
            create PVEFW-5B49FAE9 hash:net family inet hashsize 64 maxelem 64
            add PVEFW-5B49FAE9 172.16.16.0/24
    delete PVEFW-0-web-access-v6_swap (svbsoVbZ6mXJhec9puoiOqokxt0)
    
    iptables cmdlist:
    create GROUP-allow-all-clu-inte-IN (li5P61F1ja+rNYQoA2N5WToa2gA)
            -A GROUP-allow-all-clu-inte-IN -j MARK --set-mark 0x00000000/0x80000000
            -A GROUP-allow-all-clu-inte-IN -m set --match-set PVEFW-22262195 src -m set --match-set PVEFW-22262195 dst -p udp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-IN -m set --match-set PVEFW-22262195 src -m set --match-set PVEFW-22262195 dst -p tcp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-IN -m set --match-set PVEFW-45430013 src -m set --match-set PVEFW-45430013 dst -p udp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-IN -m set --match-set PVEFW-45430013 src -m set --match-set PVEFW-45430013 dst -p tcp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-IN -m set --match-set PVEFW-5B49FAE9 src -m set --match-set PVEFW-5B49FAE9 dst -p udp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-IN -m set --match-set PVEFW-5B49FAE9 src -m set --match-set PVEFW-5B49FAE9 dst -p tcp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
    create GROUP-allow-all-clu-inte-OUT (eKK2Fh5KWnCXsgwcp7++ZtJokjE)
            -A GROUP-allow-all-clu-inte-OUT -j MARK --set-mark 0x00000000/0x80000000
            -A GROUP-allow-all-clu-inte-OUT -m set --match-set PVEFW-22262195 src -m set --match-set PVEFW-22262195 dst -p udp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-OUT -m set --match-set PVEFW-22262195 src -m set --match-set PVEFW-22262195 dst -p tcp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-OUT -m set --match-set PVEFW-45430013 src -m set --match-set PVEFW-45430013 dst -p udp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-OUT -m set --match-set PVEFW-45430013 src -m set --match-set PVEFW-45430013 dst -p tcp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-OUT -m set --match-set PVEFW-5B49FAE9 src -m set --match-set PVEFW-5B49FAE9 dst -p udp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-OUT -m set --match-set PVEFW-5B49FAE9 src -m set --match-set PVEFW-5B49FAE9 dst -p tcp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
    create GROUP-allow-ssh-IN (THkm5LMvjlnBEkk9P0Cao0QA2xo)
            -A GROUP-allow-ssh-IN -j MARK --set-mark 0x00000000/0x80000000
            -A GROUP-allow-ssh-IN -m set --match-set PVEFW-0-ssh-allow-v4 src -m set --match-set PVEFW-22262195 dst -p tcp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
    create GROUP-allow-ssh-OUT (91Ie8EPx4Vl45OX4las3X6H92Yg)
            -A GROUP-allow-ssh-OUT -j MARK --set-mark 0x00000000/0x80000000
    create GROUP-allow-web-access-IN (KDFMAScU6kbH1rfJhyB6mLdU41c)
            -A GROUP-allow-web-access-IN -j MARK --set-mark 0x00000000/0x80000000
            -A GROUP-allow-web-access-IN -s 0.0.0.0/0 -m set --match-set PVEFW-22262195 dst -p tcp --sport 0:65535 --dport 8006 -g PVEFW-SET-ACCEPT-MARK
    create GROUP-allow-web-access-OUT (9GKL+2YDHkh7fCcVqpIrgwwsAUs)
            -A GROUP-allow-web-access-OUT -j MARK --set-mark 0x00000000/0x80000000
    create GROUP-allow-webcon-IN (p1xeic2kOsrJxAqb8TRlmpg/t6c)
            -A GROUP-allow-webcon-IN -j MARK --set-mark 0x00000000/0x80000000
            -A GROUP-allow-webcon-IN -s 0.0.0.0/0 -m set --match-set PVEFW-22262195 dst -p tcp --sport 0:65535 --dport 5900:5999 -g PVEFW-SET-ACCEPT-MARK
    create GROUP-allow-webcon-OUT (XMfE+Z8slYSYrrhvnLZ/dMjYobY)
            -A GROUP-allow-webcon-OUT -j MARK --set-mark 0x00000000/0x80000000
    create PVEFW-Drop (WDy2wbFe7jNYEyoO3QhUELZ4mIQ)
            -A PVEFW-Drop -p tcp --dport 43 -j PVEFW-reject
            -A PVEFW-Drop  -j PVEFW-DropBroadcast
            -A PVEFW-Drop -p icmp -m icmp --icmp-type fragmentation-needed -j ACCEPT
            -A PVEFW-Drop -p icmp -m icmp --icmp-type time-exceeded -j ACCEPT
            -A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
            -A PVEFW-Drop -p udp --match multiport --dports 135,445 -j DROP
            -A PVEFW-Drop -p udp --dport 137:139 -j DROP
            -A PVEFW-Drop -p udp --sport 137 --dport 1024:65535 -j DROP
            -A PVEFW-Drop -p tcp --match multiport --dports 135,139,445 -j DROP
            -A PVEFW-Drop -p udp --dport 1900 -j DROP
            -A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
            -A PVEFW-Drop -p udp --sport 53 -j DROP
    create PVEFW-DropBroadcast (NyjHNAtFbkH7WGLamPpdVnxHy4w)
            -A PVEFW-DropBroadcast -m addrtype --dst-type BROADCAST -j DROP
            -A PVEFW-DropBroadcast -m addrtype --dst-type MULTICAST -j DROP
            -A PVEFW-DropBroadcast -m addrtype --dst-type ANYCAST -j DROP
            -A PVEFW-DropBroadcast -d 224.0.0.0/4 -j DROP
    create PVEFW-FORWARD (qnNexOcGa+y+jebd4dAUqFSp5nw)
            -A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
            -A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
            -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-in fwln+ -j PVEFW-FWBR-IN
            -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-out fwln+ -j PVEFW-FWBR-OUT
    create PVEFW-FWBR-IN (Ijl7/xz0DD7LF91MlLCz0ybZBE0)
            -A PVEFW-FWBR-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
    create PVEFW-FWBR-OUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
    create PVEFW-HOST-IN (KL1QZfbpOxULNhvgQDE3GTXoUnE)
            -A PVEFW-HOST-IN -i lo -j ACCEPT
            -A PVEFW-HOST-IN -m conntrack --ctstate INVALID -j DROP
            -A PVEFW-HOST-IN -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
            -A PVEFW-HOST-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
            -A PVEFW-HOST-IN -p igmp -j RETURN
            -A PVEFW-HOST-IN -i vmbr0 -j GROUP-allow-webcon-IN
            -A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-IN -i vmbr0 -j GROUP-allow-web-access-IN
            -A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-IN -i vmbr0 -j GROUP-allow-ssh-IN
            -A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-IN -i eno2 -j GROUP-allow-all-clu-inte-IN
            -A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-IN -i vmbr0 -j GROUP-allow-all-clu-inte-IN
            -A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-IN -i vmbr1 -j GROUP-allow-all-clu-inte-IN
            -A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-IN -i vmbr2 -j GROUP-allow-all-clu-inte-IN
            -A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-IN -m set --match-set PVEFW-0-management-v4 src -p tcp --dport 8006 -j RETURN
            -A PVEFW-HOST-IN -m set --match-set PVEFW-0-management-v4 src -p tcp --dport 5900:5999 -j RETURN
            -A PVEFW-HOST-IN -m set --match-set PVEFW-0-management-v4 src -p tcp --dport 3128 -j RETURN
            -A PVEFW-HOST-IN -m set --match-set PVEFW-0-management-v4 src -p tcp --dport 22 -j RETURN
            -A PVEFW-HOST-IN -s 23.136.0.0/24 -d 23.136.0.0/24 -p udp --dport 5404:5405 -j RETURN
            -A PVEFW-HOST-IN -s 23.136.0.0/24 -m addrtype --dst-type MULTICAST -p udp --dport 5404:5405 -j RETURN
            -A PVEFW-HOST-IN -j PVEFW-Drop
            -A PVEFW-HOST-IN -j DROP
    create PVEFW-HOST-OUT (DCFBcfmsBikCoaH8r6OMydtDyN8)
            -A PVEFW-HOST-OUT -o lo -j ACCEPT
            -A PVEFW-HOST-OUT -m conntrack --ctstate INVALID -j DROP
            -A PVEFW-HOST-OUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
            -A PVEFW-HOST-OUT -p igmp -j RETURN
            -A PVEFW-HOST-OUT -o vmbr0 -j GROUP-allow-webcon-OUT
            -A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-OUT -o vmbr0 -j GROUP-allow-web-access-OUT
            -A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-OUT -o vmbr0 -j GROUP-allow-ssh-OUT
            -A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-OUT -o eno2 -j GROUP-allow-all-clu-inte-OUT
            -A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-OUT -o vmbr0 -j GROUP-allow-all-clu-inte-OUT
            -A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-OUT -o vmbr1 -j GROUP-allow-all-clu-inte-OUT
            -A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-OUT -o vmbr2 -j GROUP-allow-all-clu-inte-OUT
            -A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-OUT -d 23.136.0.0/24 -p tcp --dport 8006 -j RETURN
            -A PVEFW-HOST-OUT -d 23.136.0.0/24 -p tcp --dport 22 -j RETURN
            -A PVEFW-HOST-OUT -d 23.136.0.0/24 -p tcp --dport 5900:5999 -j RETURN
            -A PVEFW-HOST-OUT -d 23.136.0.0/24 -p tcp --dport 3128 -j RETURN
            -A PVEFW-HOST-OUT -d 23.136.0.0/24 -p udp --dport 5404:5405 -j RETURN
            -A PVEFW-HOST-OUT -m addrtype --dst-type MULTICAST -p udp --dport 5404:5405 -j RETURN
            -A PVEFW-HOST-OUT  -j RETURN
    create PVEFW-INPUT (+5iMmLaxKXynOB/+5xibfx7WhFk)
            -A PVEFW-INPUT -j PVEFW-HOST-IN
    create PVEFW-OUTPUT (LjHoZeSSiWAG3+2ZAyL/xuEehd0)
            -A PVEFW-OUTPUT -j PVEFW-HOST-OUT
    create PVEFW-Reject (CZJnIN6rAdpu+ej59QPr9+laMUo)
            -A PVEFW-Reject -p tcp --dport 43 -j PVEFW-reject
            -A PVEFW-Reject  -j PVEFW-DropBroadcast
            -A PVEFW-Reject -p icmp -m icmp --icmp-type fragmentation-needed -j ACCEPT
            -A PVEFW-Reject -p icmp -m icmp --icmp-type time-exceeded -j ACCEPT
            -A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
            -A PVEFW-Reject -p udp --match multiport --dports 135,445 -j PVEFW-reject
            -A PVEFW-Reject -p udp --dport 137:139 -j PVEFW-reject
            -A PVEFW-Reject -p udp --sport 137 --dport 1024:65535 -j PVEFW-reject
            -A PVEFW-Reject -p tcp --match multiport --dports 135,139,445 -j PVEFW-reject
            -A PVEFW-Reject -p udp --dport 1900 -j DROP
            -A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
            -A PVEFW-Reject -p udp --sport 53 -j DROP
    create PVEFW-SET-ACCEPT-MARK (Hg/OIgIwJChBUcWU8Xnjhdd2jUY)
            -A PVEFW-SET-ACCEPT-MARK  -j MARK --set-mark 0x80000000/0x80000000
    create PVEFW-logflags (MN4PH1oPZeABMuWr64RrygPfW7A)
            -A PVEFW-logflags  -j DROP
    create PVEFW-reject (Jlkrtle1mDdtxDeI9QaDSL++Npc)
            -A PVEFW-reject -m addrtype --dst-type BROADCAST -j DROP
            -A PVEFW-reject -s 224.0.0.0/4 -j DROP
            -A PVEFW-reject -p icmp -j DROP
            -A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
            -A PVEFW-reject -p udp -j REJECT --reject-with icmp-port-unreachable
            -A PVEFW-reject -p icmp -j REJECT --reject-with icmp-host-unreachable
            -A PVEFW-reject  -j REJECT --reject-with icmp-host-prohibited
    create PVEFW-smurflog (2gfT1VMkfr0JL6OccRXTGXo+1qk)
            -A PVEFW-smurflog  -j DROP
    create PVEFW-smurfs (HssVe5QCBXd5mc9kC88749+7fag)
            -A PVEFW-smurfs -s 0.0.0.0/32 -j RETURN
            -A PVEFW-smurfs -m addrtype --src-type BROADCAST -g PVEFW-smurflog
            -A PVEFW-smurfs -s 224.0.0.0/4 -g PVEFW-smurflog
    create PVEFW-tcpflags (CMFojwNPqllyqD67NeI5m+bP5mo)
            -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
            -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
            -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
            -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
            -A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags
    
    ip6tables cmdlist:
    create GROUP-allow-all-clu-inte-IN (xKHHwMLiyn3uuYAYekG3tdp8d4o)
            -A GROUP-allow-all-clu-inte-IN -j MARK --set-mark 0x00000000/0x80000000
            -A GROUP-allow-all-clu-inte-IN -m set --match-set PVEFW-20261E6F src -m set --match-set PVEFW-20261E6F dst -p udp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-IN -m set --match-set PVEFW-20261E6F src -m set --match-set PVEFW-20261E6F dst -p tcp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-IN -m set --match-set PVEFW-47430339 src -m set --match-set PVEFW-47430339 dst -p udp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-IN -m set --match-set PVEFW-47430339 src -m set --match-set PVEFW-47430339 dst -p tcp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-IN -m set --match-set PVEFW-5949F7C3 src -m set --match-set PVEFW-5949F7C3 dst -p udp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-IN -m set --match-set PVEFW-5949F7C3 src -m set --match-set PVEFW-5949F7C3 dst -p tcp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
    create GROUP-allow-all-clu-inte-OUT (HecF52nVHoapeChnMfXCor78Yrs)
            -A GROUP-allow-all-clu-inte-OUT -j MARK --set-mark 0x00000000/0x80000000
            -A GROUP-allow-all-clu-inte-OUT -m set --match-set PVEFW-20261E6F src -m set --match-set PVEFW-20261E6F dst -p udp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-OUT -m set --match-set PVEFW-20261E6F src -m set --match-set PVEFW-20261E6F dst -p tcp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-OUT -m set --match-set PVEFW-47430339 src -m set --match-set PVEFW-47430339 dst -p udp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-OUT -m set --match-set PVEFW-47430339 src -m set --match-set PVEFW-47430339 dst -p tcp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-OUT -m set --match-set PVEFW-5949F7C3 src -m set --match-set PVEFW-5949F7C3 dst -p udp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
            -A GROUP-allow-all-clu-inte-OUT -m set --match-set PVEFW-5949F7C3 src -m set --match-set PVEFW-5949F7C3 dst -p tcp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
    create GROUP-allow-ssh-IN (KdfBTKcpYU/pa0bbZ3CxWgEQqNQ)
            -A GROUP-allow-ssh-IN -j MARK --set-mark 0x00000000/0x80000000
            -A GROUP-allow-ssh-IN -m set --match-set PVEFW-0-ssh-allow-v6 src -m set --match-set PVEFW-20261E6F dst -p tcp --sport 0:65535 --dport 0:65535 -g PVEFW-SET-ACCEPT-MARK
    create GROUP-allow-ssh-OUT (91Ie8EPx4Vl45OX4las3X6H92Yg)
            -A GROUP-allow-ssh-OUT -j MARK --set-mark 0x00000000/0x80000000
    create GROUP-allow-web-access-IN (hyhiV7mbj2sbFmWiFA1S49J/Jjc)
            -A GROUP-allow-web-access-IN -j MARK --set-mark 0x00000000/0x80000000
    create GROUP-allow-web-access-OUT (9GKL+2YDHkh7fCcVqpIrgwwsAUs)
            -A GROUP-allow-web-access-OUT -j MARK --set-mark 0x00000000/0x80000000
    create GROUP-allow-webcon-IN (P/x2MgJ9EvCI48OoI+VgIUWIPWM)
            -A GROUP-allow-webcon-IN -j MARK --set-mark 0x00000000/0x80000000
    create GROUP-allow-webcon-OUT (XMfE+Z8slYSYrrhvnLZ/dMjYobY)
            -A GROUP-allow-webcon-OUT -j MARK --set-mark 0x00000000/0x80000000
    create PVEFW-Drop (Jb79Uw7z1vZglIcV7QXA5uY/nbk)
            -A PVEFW-Drop -p tcp --dport 43 -j PVEFW-reject
            -A PVEFW-Drop  -j PVEFW-DropBroadcast
            -A PVEFW-Drop -p icmpv6 -m icmpv6 --icmpv6-type destination-unreachable -j ACCEPT
            -A PVEFW-Drop -p icmpv6 -m icmpv6 --icmpv6-type time-exceeded -j ACCEPT
            -A PVEFW-Drop -p icmpv6 -m icmpv6 --icmpv6-type packet-too-big -j ACCEPT
            -A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
            -A PVEFW-Drop -p udp --match multiport --dports 135,445 -j DROP
            -A PVEFW-Drop -p udp --dport 137:139 -j DROP
            -A PVEFW-Drop -p udp --sport 137 --dport 1024:65535 -j DROP
            -A PVEFW-Drop -p tcp --match multiport --dports 135,139,445 -j DROP
            -A PVEFW-Drop -p udp --dport 1900 -j DROP
            -A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
            -A PVEFW-Drop -p udp --sport 53 -j DROP
    create PVEFW-DropBroadcast (8Krk5Nh8pDZOOc7BQAbM6PlyFSU)
            -A PVEFW-DropBroadcast -d ff00::/8 -j DROP
    create PVEFW-FORWARD (qnNexOcGa+y+jebd4dAUqFSp5nw)
            -A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
            -A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
            -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-in fwln+ -j PVEFW-FWBR-IN
            -A PVEFW-FORWARD -m physdev --physdev-is-bridged --physdev-out fwln+ -j PVEFW-FWBR-OUT
    create PVEFW-FWBR-IN (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
    create PVEFW-FWBR-OUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
    create PVEFW-HOST-IN (x5eQwnSY7fnOd5J74kU8QVGyDdM)
            -A PVEFW-HOST-IN -i lo -j ACCEPT
            -A PVEFW-HOST-IN -m conntrack --ctstate INVALID -j DROP
            -A PVEFW-HOST-IN -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
            -A PVEFW-HOST-IN -p icmpv6 --icmpv6-type router-solicitation -j RETURN
            -A PVEFW-HOST-IN -p icmpv6 --icmpv6-type router-advertisement -j RETURN
            -A PVEFW-HOST-IN -p icmpv6 --icmpv6-type neighbor-solicitation -j RETURN
            -A PVEFW-HOST-IN -p icmpv6 --icmpv6-type neighbor-advertisement -j RETURN
            -A PVEFW-HOST-IN -p igmp -j RETURN
            -A PVEFW-HOST-IN -i vmbr0 -j GROUP-allow-webcon-IN
            -A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-IN -i vmbr0 -j GROUP-allow-web-access-IN
            -A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-IN -i vmbr0 -j GROUP-allow-ssh-IN
            -A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-IN -i eno2 -j GROUP-allow-all-clu-inte-IN
            -A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-IN -i vmbr0 -j GROUP-allow-all-clu-inte-IN
            -A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-IN -i vmbr1 -j GROUP-allow-all-clu-inte-IN
            -A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-IN -i vmbr2 -j GROUP-allow-all-clu-inte-IN
            -A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-IN -m set --match-set PVEFW-0-management-v6 src -p tcp --dport 8006 -j RETURN
            -A PVEFW-HOST-IN -m set --match-set PVEFW-0-management-v6 src -p tcp --dport 5900:5999 -j RETURN
            -A PVEFW-HOST-IN -m set --match-set PVEFW-0-management-v6 src -p tcp --dport 3128 -j RETURN
            -A PVEFW-HOST-IN -m set --match-set PVEFW-0-management-v6 src -p tcp --dport 22 -j RETURN
            -A PVEFW-HOST-IN -j PVEFW-Drop
            -A PVEFW-HOST-IN -j DROP
    create PVEFW-HOST-OUT (FIWwwvGWMx8qX29NVy07ANJ9hmQ)
            -A PVEFW-HOST-OUT -o lo -j ACCEPT
            -A PVEFW-HOST-OUT -m conntrack --ctstate INVALID -j DROP
            -A PVEFW-HOST-OUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
            -A PVEFW-HOST-OUT -p icmpv6 --icmpv6-type router-solicitation -j RETURN
            -A PVEFW-HOST-OUT -p icmpv6 --icmpv6-type neighbor-solicitation -j RETURN
            -A PVEFW-HOST-OUT -p icmpv6 --icmpv6-type neighbor-advertisement -j RETURN
            -A PVEFW-HOST-OUT -p igmp -j RETURN
            -A PVEFW-HOST-OUT -o vmbr0 -j GROUP-allow-webcon-OUT
            -A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-OUT -o vmbr0 -j GROUP-allow-web-access-OUT
            -A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-OUT -o vmbr0 -j GROUP-allow-ssh-OUT
            -A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-OUT -o eno2 -j GROUP-allow-all-clu-inte-OUT
            -A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-OUT -o vmbr0 -j GROUP-allow-all-clu-inte-OUT
            -A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-OUT -o vmbr1 -j GROUP-allow-all-clu-inte-OUT
            -A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-OUT -o vmbr2 -j GROUP-allow-all-clu-inte-OUT
            -A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
            -A PVEFW-HOST-OUT  -j RETURN
    create PVEFW-INPUT (+5iMmLaxKXynOB/+5xibfx7WhFk)
            -A PVEFW-INPUT -j PVEFW-HOST-IN
    create PVEFW-OUTPUT (LjHoZeSSiWAG3+2ZAyL/xuEehd0)
            -A PVEFW-OUTPUT -j PVEFW-HOST-OUT
    create PVEFW-Reject (aL1nrxJk/u3XmTb3Am2eaM/3yCM)
            -A PVEFW-Reject -p tcp --dport 43 -j PVEFW-reject
            -A PVEFW-Reject  -j PVEFW-DropBroadcast
            -A PVEFW-Reject -p icmpv6 -m icmpv6 --icmpv6-type destination-unreachable -j ACCEPT
            -A PVEFW-Reject -p icmpv6 -m icmpv6 --icmpv6-type time-exceeded -j ACCEPT
            -A PVEFW-Reject -p icmpv6 -m icmpv6 --icmpv6-type packet-too-big -j ACCEPT
            -A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
            -A PVEFW-Reject -p udp --match multiport --dports 135,445 -j PVEFW-reject
            -A PVEFW-Reject -p udp --dport 137:139 -j PVEFW-reject
            -A PVEFW-Reject -p udp --sport 137 --dport 1024:65535 -j PVEFW-reject
            -A PVEFW-Reject -p tcp --match multiport --dports 135,139,445 -j PVEFW-reject
            -A PVEFW-Reject -p udp --dport 1900 -j DROP
            -A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
            -A PVEFW-Reject -p udp --sport 53 -j DROP
    create PVEFW-SET-ACCEPT-MARK (Hg/OIgIwJChBUcWU8Xnjhdd2jUY)
            -A PVEFW-SET-ACCEPT-MARK  -j MARK --set-mark 0x80000000/0x80000000
    create PVEFW-logflags (MN4PH1oPZeABMuWr64RrygPfW7A)
            -A PVEFW-logflags  -j DROP
    create PVEFW-reject (TeZhczhc17LK2pqE7UkGmRMJLNU)
            -A PVEFW-reject -p icmpv6 -j DROP
            -A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
    create PVEFW-tcpflags (CMFojwNPqllyqD67NeI5m+bP5mo)
            -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
            -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
            -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
            -A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
            -A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags
    
    ebtables cmdlist:
    create PVEFW-FORWARD (ULtZ6lqjrD/jAKLY+OZo3BbXs9k)
            -A PVEFW-FORWARD -p IPv4 -j ACCEPT
            -A PVEFW-FORWARD -p IPv6 -j ACCEPT
            -A PVEFW-FORWARD -o fwln+ -j PVEFW-FWBR-OUT
    create PVEFW-FWBR-OUT (2jmj7l5rSw0yVb/vlWAYkK/YBwk)
    detected changes
    root@he-s01-r01-pve01:~#
    
    Note my pve-firewall localnet shows 23.136.0.0/24 which is the same as my vm subnet and my pve nodes i have rules restiricting that network like ssh and other ports from public ips. So will my rules overide the pve localnet rules i hope?

    Any ideas on how to get mycluster firewall running?
     
    #1 HE_Cole, Dec 1, 2018
    Last edited: Dec 2, 2018
  2. spirit

    spirit Well-Known Member
    Proxmox VE Subscriber

    Joined:
    Apr 2, 2010
    Messages:
    3,196
    Likes Received:
    110
    cluster firewall rules only apply to host node (proxmox management), this is iptables INPUT/OUPUT but not on vms. (iptables FORWARD).

    This is the same than make rules for each host.


    >>Question? Do i need a explict deny any any rule in my rules to block traffic that does not match a rule?
    already done . "My data center input policy is DROP"


    >>Do i need any thing speical for corosync like allow multicast ip?
    they are default rules for communications inter host (including corosync, ssh 22, 8006, ...)
    you need to add rules for ceph.


    #iptables-save to show current rules (look INPUT/OUPUT chains)
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    HE_Cole likes this.
  3. HE_Cole

    HE_Cole Member
    Proxmox VE Subscriber

    Joined:
    Oct 25, 2018
    Messages:
    34
    Likes Received:
    0
    spirit Thanks !

    Between my pve hosts networks like "ceph,corosync etc'' i allow any/any between there seprate subnets.

    My output of iptables-save is this,
    Code:
    # Generated by iptables-save v1.6.0 on Sun Dec  2 15:00:02 2018
    *filter
    :INPUT ACCEPT [613871423:839715363079]
    :FORWARD ACCEPT [270995:271807649]
    :OUTPUT ACCEPT [602912745:747533392365]
    COMMIT
    # Completed on Sun Dec  2 15:00:02 2018
    I also noticed it only created managegment rules for my 23.136.0.0/24 subnet. My ceph and corosync are on different subnets and interfaces. But i created rules for all of them any ways separately in datacenter -> firewall -> groups.

    Since i created my rules in datacenter -> firewall it should override the auto created management rules right???


    You answered alot of my questions but my firewall is still not applying.
    I created all my rules from the GUI and had no issues but the firewall wont take effect.

    Any ideas on why my firewall is not applying?
     
    #3 HE_Cole, Dec 2, 2018
    Last edited: Dec 2, 2018
  4. spirit

    spirit Well-Known Member
    Proxmox VE Subscriber

    Joined:
    Apr 2, 2010
    Messages:
    3,196
    Likes Received:
    110
    can you post your cluster.fw && host.fw config files ?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. HE_Cole

    HE_Cole Member
    Proxmox VE Subscriber

    Joined:
    Oct 25, 2018
    Messages:
    34
    Likes Received:
    0
    Sure !!!

    Hereis cluster.fw
    Code:
    [OPTIONS]
    
    enable: 1
    
    [ALIASES]
    
    all-ipv6 ::/0
    all-ipv4 0.0.0.0/0
    
    [IPSET ceph-cluster-pve01]
    
    10.10.11.0/24
    10.10.12.0/24
    
    [IPSET corosync-cluster-pve01]
    
    172.16.16.0/24
    
    [IPSET ssh-allow]
    
    104.15.112.69
    
    [IPSET wan-cluster-pve01]
    
    23.136.0.1
    23.136.0.3
    23.136.0.4
    23.136.0.5
    23.136.0.6
    23.136.0.7
    2606:8d80::/32
    
    [IPSET web-access]
    
    all-ipv4
    all-ipv6
    
    [RULES]
    
    GROUP allow-webcon -i vmbr0
    GROUP allow-web-access -i vmbr0
    GROUP allow-ssh -i vmbr0
    GROUP allow-all-clu-inte -i eno2 # COROSYNC
    GROUP allow-all-clu-inte -i vmbr0 # WAN
    GROUP allow-all-clu-inte -i vmbr1 # CEPH
    GROUP allow-all-clu-inte -i vmbr2 # CEPH
    |GROUP block-all -i vmbr0
    
    [group allow-all-clu-inte]
    
    IN ACCEPT -source 224.0.0.0/4 -dest 224.0.0.0/4 -p tcp -dport 0:65535 -sport 0:65535
    OUT ACCEPT -source 224.0.0.0/4 -dest 224.0.0.0/4 -p tcp -dport 0:65535 -sport 0:65535
    OUT ACCEPT -source 224.0.0.0/4 -dest 224.0.0.0/4 -p udp -dport 0:65535 -sport 0:65535
    IN ACCEPT -source 224.0.0.0/4 -dest 224.0.0.0/4 -p udp -dport 0:65535 -sport 0:65535
    IN ACCEPT -source +wan-cluster-pve01 -dest +wan-cluster-pve01 -p udp -dport 0:65535 -sport 0:65535
    OUT ACCEPT -source +wan-cluster-pve01 -dest +wan-cluster-pve01 -p udp -dport 0:65535 -sport 0:65535
    IN ACCEPT -source +wan-cluster-pve01 -dest +wan-cluster-pve01 -p tcp -dport 0:65535 -sport 0:65535
    OUT ACCEPT -source +wan-cluster-pve01 -dest +wan-cluster-pve01 -p tcp -dport 0:65535 -sport 0:65535
    IN ACCEPT -source +ceph-cluster-pve01 -dest +ceph-cluster-pve01 -p udp -dport 0:65535 -sport 0:65535
    OUT ACCEPT -source +ceph-cluster-pve01 -dest +ceph-cluster-pve01 -p udp -dport 0:65535 -sport 0:65535
    IN ACCEPT -source +ceph-cluster-pve01 -dest +ceph-cluster-pve01 -p tcp -dport 0:65535 -sport 0:65535
    OUT ACCEPT -source +ceph-cluster-pve01 -dest +ceph-cluster-pve01 -p tcp -dport 0:65535 -sport 0:65535
    IN ACCEPT -source +corosync-cluster-pve01 -dest +corosync-cluster-pve01 -p udp -dport 0:65535 -sport 0:65535
    OUT ACCEPT -source +corosync-cluster-pve01 -dest +corosync-cluster-pve01 -p udp -dport 0:65535 -sport 0:65535
    IN ACCEPT -source +corosync-cluster-pve01 -dest +corosync-cluster-pve01 -p tcp -dport 0:65535 -sport 0:65535
    OUT ACCEPT -source +corosync-cluster-pve01 -dest +corosync-cluster-pve01 -p tcp -dport 0:65535 -sport 0:65535
    
    
    [group allow-all-in-out]
    
    OUT ACCEPT -source 0.0.0.0/0 -dest 0.0.0.0/0 -p icmp # Allow-All OUT ICMP v4
    IN ACCEPT -source 0.0.0.0/0 -dest 0.0.0.0/0 -p icmp # Allow-All IN ICMP v4
    OUT ACCEPT -source ::/0 -dest ::/0 -p udp -dport 0:65535 -sport 0:65535 # Allow-All OUT UDP v6
    OUT ACCEPT -source 0.0.0.0/0 -dest 0.0.0.0/0 -p udp -dport 0:65535 -sport 0:65535 # Allow-All OUT UDP v4
    OUT ACCEPT -source ::/0 -dest ::/0 -p tcp -dport 0:65535 -sport 0:65535 # Allow-All OUT TCP v6
    OUT ACCEPT -source 0.0.0.0/0 -dest 0.0.0.0/0 -p tcp -dport 0:65535 -sport 0:65535 # Allow-All OUT TCP v4
    IN ACCEPT -source ::/0 -dest ::/0 -p udp -dport 0:65535 -sport 0:65535 # Allow-All IN UDP v6
    IN ACCEPT -source 0.0.0.0/0 -dest 0.0.0.0/0 -p udp -dport 0:65535 -sport 0:65535 # Allow-All IN UDP v4
    IN ACCEPT -source ::/0 -dest ::/0 -p tcp -dport 0:65535 -sport 0:65535 # Allow-All IN TCP v6
    IN ACCEPT -source 0.0.0.0/0 -dest 0.0.0.0/0 -p tcp -dport 0:65535 -sport 0:65535 # Allow-All IN TCP v4
    
    [group allow-ssh]
    
    IN ACCEPT -source +ssh-allow -dest +wan-cluster-pve01 -p tcp -dport 0:65535 -sport 0:65535
    
    [group allow-web-access]
    
    IN ACCEPT -source all-ipv4 -dest +wan-cluster-pve01 -p tcp -dport 8006 -sport 0:65535
    
    [group allow-webcon]
    
    IN ACCEPT -source all-ipv4 -dest +wan-cluster-pve01 -p tcp -dport 5900:5999 -sport 0:65535
    
    [group block-all]
    
    IN DROP -source all-ipv4 -dest all-ipv4 -p tcp -dport 0:65535 -sport 0:65535
    
    
    
    
    The host.fw does not exist.

    ls /etc/pve/nodes/he-s01-r01-pve01
    lrm_status openvz pveproxy-ssl.key pve-ssl.key qemu-server
    lxc priv pveproxy-ssl.pem pve-ssl.pem
    Are the only directorys.

    Thanks a bunch.
     
  6. spirit

    spirit Well-Known Member
    Proxmox VE Subscriber

    Joined:
    Apr 2, 2010
    Messages:
    3,196
    Likes Received:
    110
    I'm not sure port 0 is possible.

    BTW, your rules can be easily better like

    >>GROUP allow-all-clu-inte -i eno2 # COROSYNC
    >>GROUP allow-all-clu-inte -i vmbr0 # WAN
    >>GROUP allow-all-clu-inte -i vmbr1 # CEPH
    >>GROUP allow-all-clu-inte -i vmbr2 # CEPH

    why not simply do
    IN ACCEPT eno2
    IN ACCEPT vmbr0
    ...

    ?

    if you really want to keep group

    something like this:
    > IN ACCEPT -source 224.0.0.0/4 -dest 224.0.0.0/4 -p tcp -dport 0:65535 -sport 0:65535

    IN ACCEPT -source 224.0.0.0/4 -dest 224.0.0.0/4
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  7. HE_Cole

    HE_Cole Member
    Proxmox VE Subscriber

    Joined:
    Oct 25, 2018
    Messages:
    34
    Likes Received:
    0
    Hey again! I agree they look complex and well they are, But i plan on cleaning them up later. I changed all the 0 to 1 and still the firewall is not applying still states
    Status: enabled/running (pending changes) and all my rules show active but none of them are applyed.

    I wish it would give a error so i could see what the problem is.

    1. Do i need a deny any any at the end or is a deny all implyed if a packet does not match a rule?

    2. Will my rules overide the local firewall rules pve creates as i need them too.

    3. When i am making a rule in the PVE GUI and i dont set a protocal or a port is that the same as ANY port/protocal?

    I am using the PVE GUI to to edit and make rules.

    My goal is simple.

    Block all incoming ports to nodes ip's except for ports, 8006, 5900-5999, and 3128
    Allow only listed ips to access node SSH.
    Allow all in/out on any ports from only node to node public IP's.

    I am going to delete all my rules and start over.

    NOTE, all my nodes have a separate public ip. corosync and ceph have there own private ip's and interfaces.

    Any ideas ?
     
    #7 HE_Cole, Dec 4, 2018
    Last edited: Dec 4, 2018
  8. HE_Cole

    HE_Cole Member
    Proxmox VE Subscriber

    Joined:
    Oct 25, 2018
    Messages:
    34
    Likes Received:
    0
    I would just like to add i deleted all my firewall rules from the GUI and just made a simple rule to block/allow ssh from a list of IP's and it still wont apply it. I made all my rules from the pve gui and and enabled them and still if i run firewall status i get enabled/running (pending changes) and none of my ports i blocked are blocked. I even just made a simple block all rule and placed it at the top to test and it did nothing. Becuase my corosync and ceph are on sepreate networks i always create allow rules for them so they dont get blocked and all host to host communication is allowed.

    What can i do?

    My goal is to block ssh to all nodes and allow ssh from only listed ips.
    Allow all in for PVE GUI and VNC on all nodes.
    All other incoming ports on wan need to be blocked.


    Any help would be very much appreciated.
     
  9. spirit

    spirit Well-Known Member
    Proxmox VE Subscriber

    Joined:
    Apr 2, 2010
    Messages:
    3,196
    Likes Received:
    110
    >>Block all incoming ports to nodes ip's except for ports, 8006, 5900-5999, and 3128
    >>Allow only listed ips to access node SSH.
    >>Allow all in/out on any ports from only node to node public IP's.

    by default proxmox open ports 8006, 5900-5999, and 3128,ssh + multicast for the subnet of the proxmox management ip.



    simply (on each host or at datacenter level),

    default IN : DROP or REJECT
    default OUT: OPEN

    then, in rules

    >>Block all incoming ports to nodes ip's except for ports, 8006, 5900-5999, and 3128

    IN allow tcp/8066 ( src ...ip or network subnet)
    IN allow tcp/5900 ( src ...ip or network subnet)
    IN allow tcp/2128 ( src ...ip or network subnet)

    >>Allow only listed ips to access node SSH.
    IN allow tcp/22 src youripset...

    >>Allow all in/out on any ports from only node to node public IP's.
    by default proxmox open ports 8006, 5900-5999, and 3128,ssh + multicast for the subnet of the proxmox management ip.

    but this should work

    IN allow src ipsetwithyournodepublicip




    (Note that rules at datacenter/host level, only apply to your proxmox management ip (iptables INPUT/OUTPUT), not on vms (iptables FORWARD))
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    HE_Cole likes this.
  10. HE_Cole

    HE_Cole Member
    Proxmox VE Subscriber

    Joined:
    Oct 25, 2018
    Messages:
    34
    Likes Received:
    0
    Sounds good Spirit!

    Few last 3 questions, Since promox only opens ssh,multicast etc on the management subnet in my case thats 23.136.0.0/24 which is the same network as my VM's too it wont cause issues with vms will it? And my corosync and ceph are on sepreate private subnets do i need to create rules to allow multicast/corosync 172.16.16.0/128 and ceph 10.11.11.0/24 on there interfaces and IP's since my cluster does not run corosync and ceph on the management network and pve does not know to allow traffic for them?

    Do add the rules to the phsical interface or the bridge?

    I was able to get it working, Turns out it was not activating because the local_management section on iptables was not auto generated by Promox i manully added it and it works now.



    Thanks
     
    #10 HE_Cole, Dec 5, 2018
    Last edited: Dec 7, 2018
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice