How can I change chain do-reject to reject?

[encryptedserver]

The last rule in chain do-reject is wrong. I was wondering why the connection keeps timing out instead of immediately getting a reject.

It is currently at drop when it should be reject, even the rule name is do-reject!

chain do-reject {
    meta pkttype broadcast drop
    ip saddr 224.0.0.0/4 drop
    meta l4proto tcp reject with tcp reset
    meta l4proto { icmp, ipv6-icmp } reject
    reject with icmp host-prohibited
    reject with icmpv6 admin-prohibited
    drop
}

Please fix the rule by replacing drop with reject.

Is there any way to edit the file manually right now myself? where is the nftables rules stored in PVE?

 

[shanreich]

The rule is fine - those 2 statements should be terminal already:

    reject with icmp host-prohibited
    reject with icmpv6 admin-prohibited

The only way the drop is reached if it is an ARP packet in the bridge table. The drop rule is just as a safety measure in case there is any unforeseen traffic that cannot be matched / rejected by nftables.



Are you by chance trying to reject incoming traffic of a guest? This is currently not possible due to limitations of nftables, we also have a small section in the documentation [1] regarding that:

• REJECT is currently not possible for guest traffic (traffic will instead be dropped).

[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_nftables

 

[encryptedserver]

I added the reject rule to block local lan

        set v4-dc/lan {
                type ipv4_addr
                flags interval
                elements = { 10.0.0.0/8, 172.16.0.0/12,
                             192.168.0.0/16 }
        }

        set v4-dc/lan-nomatch {
                type ipv4_addr
                flags interval
        }

        set v6-dc/lan {
                type ipv6_addr
                flags interval
                elements = { fc00::/7 }
        }

        chain group-deny-lan-access-out {                                                                                                                 
                ip daddr @v4-dc/gateway ip daddr != @v4-dc/gateway-nomatch accept                                                                         
                ip6 daddr @v6-dc/gateway ip6 daddr != @v6-dc/gateway-nomatch accept                                                                       
                ip daddr @v4-dc/lan ip daddr != @v4-dc/lan-nomatch jump do-reject                                                                         
                ip6 daddr @v6-dc/lan ip6 daddr != @v6-dc/lan-nomatch jump do-reject                                                                       
        }

For example if I ping 10.88.88.12 from my VM it should reject immediately but I am getting a timeout.

 

[shanreich]

I've looked at this, the issue here seems to be that the packet is actually rejected, but the responding packet has CT state invalid - so it gets dropped by the incoming firewall:

trace id c784a776 bridge proxmox-firewall-guests vm-in packet: oif "tap2001i0" ether saddr 36:86:46:c7:59:3f ether daddr bc:24:11:5e:44:b4 ip saddr 10.101.1.1 ip daddr 10.101.95.1 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 0 ip protocol icmp ip length 112 icmp type destination-unreachable icmp code port-unreachable icmp id 0 icmp sequence 0 @th,64,96 0x4500005419dc40004001ac01
trace id c784a776 bridge proxmox-firewall-guests vm-in rule ether type != arp ct state vmap { invalid : drop, established : accept, related : accept } (verdict drop)

It might make sense to "whitelist" certain ICMP types but I'll have to think about the possible implications of this - but I'll create a patch once I've looked more into this. Some ICMP type certainly should be (e.g. the ones used for Path MTU discovery)

 

[shanreich]

You should be able to work around this for now by creating a rule that exempts ICMP packets with type destination unreachable from conntrack:

#!/usr/sbin/nft -f
table bridge icmp-reject
delete table bridge icmp-reject

table bridge icmp-reject {
    chain postrouting {
        type filter hook postrouting priority -350; policy accept;
        oifname "tap*" icmp type destination-unreachable notrack
    }
}

As well as explicitly allowing ICMP packets with type destination unreachable in the incoming chain of the VM:

[RULES]

IN ACCEPT -p icmp -log nolog -icmp-type destination-unreachable

 

[encryptedserver]

#!/usr/sbin/nft -f
table bridge icmp-reject
delete table bridge icmp-reject

table bridge icmp-reject {
chain postrouting {
type filter hook postrouting priority -350; policy accept;
oifname "tap*" icmp type destination-unreachable notrack
}
}

I tried pasting the code in /etc/nftables.conf but it does not work over there, where should I paste these commands?

 

[shanreich]

I tried pasting the code in /etc/nftables.conf but it does not work over there, where should I paste these commands?

You can just save it as a new file, e.g. icmp.nft, and then make it executable (chmod +x icmp.nft) and then execute it (./icmp.nft)

 

[encryptedserver]

You can just save it as a new file, e.g. icmp.nft, and then make it executable (chmod +x icmp.nft) and then execute it (./icmp.nft)

Looks like I might have asked the wrong question and the above solutions can be ignored. Don't even worry about them. I will try to ask the question again in simple terms.

I created two firewall rules in a group and applied to my all my VMs

Group name: Block LAN
Rule1: out - 192.168.1.1 / fd88::1 - ACCEPT
Rule2: out - 192.168.1.1/24 / fd88::/64 - REJECT

While the above rule works and now VMs cannot access any other devices in the network but testing the connection times out instead of immediate REJECT which is in the rule.

I am not sure how to get the REJECT rule to work and immediately get the answer that the connection is blocked instead of timing out after several minutes.

Hope anyone else can test this common firewall rule.

 

[spirit]

Looks like I might have asked the wrong question and the above solutions can be ignored. Don't even worry about them. I will try to ask the question again in simple terms.

I created two firewall rules in a group and applied to my all my VMs

Group name: Block LAN
Rule1: out - 192.168.1.1 / fd88::1 - ACCEPT
Rule2: out - 192.168.1.1/24 / fd88::/64 - REJECT

While the above rule works and now VMs cannot access any other devices in the network but testing the connection times out instead of immediate REJECT which is in the rule.

I am not sure how to get the REJECT rule to work and immediately get the answer that the connection is blocked instead of timing out after several minutes.

Hope anyone else can test this common firewall rule.

The problem is that we can't do a reject with a bridged firewall here with nftables. (I was working with iptables with some dirty trick, not available in nftables).

 

[shanreich]

I am not sure how to get the REJECT rule to work and immediately get the answer that the connection is blocked instead of timing out after several minutes.

For outgoing connections it should work with the nftables rules I posted above. For incoming it doesn't.

The reason for this is that the ICMP packet with 'Destination Unreachable' that is generated by the firewall gets dropped because it has CT state invalid. The rule I posted above fixes that temporarily by exempting all ICMP 'Destination Unreachable' from conntrack.

We also recently merged a patch that should address this issue [1] - it's not yet been released though.

[1] https://git.proxmox.com/?p=proxmox-firewall.git;a=commit;h=1e16cc8827545ab209448f3f1933a0422f5b9dda

 

[spirit]

For outgoing connections it should work with the nftables rules I posted above. For incoming it doesn't.

ah yes, sorry, I miss read it.

The reason for this is that the ICMP packet with 'Destination Unreachable' that is generated by the firewall gets dropped because it has CT state invalid.

make sense.

The rule I posted above fixes that temproarily.

We also recently merged a patch that should address this issue [1] - it's not yet been released though.

[1] https://git.proxmox.com/?p=proxmox-firewall.git;a=commit;h=1e16cc8827545ab209448f3f1933a0422f5b9dda

thanks! (I'll do tests with nftables next month)

 

[encryptedserver]

I will also test this when this is available. I also added a rule to every VM - IN ACCEPT, which allows everything inbound.

 

[encryptedserver]

Looks like this has been fixed in 8.3, I am getting a proper REJECT now for outgoing connections (LAN access is blocked).