VM lost internet connection when firewall on

[cherimos]

Hi everyone,

I'm encountering an issue where my host-level iptables NAT rule (for VMs on a private bridge to access the internet) stops working when I enable the Proxmox VE firewall on the VM's network interface.

Setup:

• Proxmox VE Host - Dedicated server
• VMs are on a private bridge vmbr1 (e.g., network 192.168.3.0/24, VM IP 192.168.3.2, vmbr1 IP 192.168.3.1).
• Host has a public bridge vmbr0 (e.g., IP 78.46.xx.xx) for internet access.
• net.ipv4.ip_forward is enabled on the host.

Goal:I want my VMs to access the internet (which requires NAT on the host) AND I want to use the Proxmox VE firewall (enabled on the VM's NIC in the PVE GUI) for filtering and security.

Observations:

Scenario 1: Proxmox VE Firewall for VM NIC is OFF

1. I have a MASQUERADE rule in /etc/network/interfaces for vmbr1's post-up:iptables -t nat -A POSTROUTING -s 192.168.3.0/24 -o vmbr0 -j MASQUERADE
2. The "Firewall" option for the VM's network device in the PVE GUI is unchecked (OFF).
3. Result: Internet access for the VM works perfectly.
   • tcpdump on the host's vmbr0 shows packets leaving with the public IP of vmbr0.
   • The packet/byte counters for the MASQUERADE rule in iptables -t nat -L POSTROUTING -v -n increment as expected.

Scenario 2: Proxmox VE Firewall for VM NIC is ON

1. The same MASQUERADE rule (or an equivalent SNAT --to-source <public_IP> rule) is present in the host's POSTROUTING chain (verified with iptables -t nat -L POSTROUTING -v -n).
2. The "Firewall" option for the VM's network device in the PVE GUI is checked (ON).
3. I have added ACCEPT OUT rules in the PVE firewall GUI for the VM (e.g., allow all outbound from 192.168.3.0/24 for testing).
4. Result: Internet access for the VM FAILS.
   • tcpdump on the host's vmbr0 shows packets leaving with the VM's private source IP (e.g., 192.168.3.2).
   • The packet/byte counters for the MASQUERADE (or SNAT) rule in iptables -t nat -L POSTROUTING -v -n remain at 0 or do not increment, indicating the rule is not being matched.

Question:Why does enabling the Proxmox VE firewall on a VM's network interface prevent my standard host-level POSTROUTING NAT rule (which is confirmed to be syntactically correct as it works when PVE FW is off) from matching or triggering? The packets are clearly being forwarded by the PVE firewall (as seen by tcpdump on vmbr0), but they are not being NATted by my host rule.

Is there a recommended way to configure outbound NAT for VMs when the Proxmox VE NIC-specific firewall is active? I couldn't find a clear SNAT/DNAT configuration section under Datacenter -> Firewall in my PVE GUI for this purpose (or I might be looking in the wrong place for this specific use case). How can I achieve both PVE firewalling for the VM and working NAT?

Any insights or suggestions would be greatly appreciated!

This is my interface at the moment:

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

iface lo inet6 loopback

#auto enp5s0
iface enp5s0 inet manual

auto enp5s0.4000
#pre-up modprobe 8021q
iface enp5s0.4000 inet static
address 192.168.1.2/24
mtu 1400

auto vmbr0
iface vmbr0 inet static
address 78.46.xx.xx/27
gateway 78.46.xx.xx
bridge-ports enp5s0
bridge-stp off
bridge-fd 1
bridge-vlan-aware yes
bridge-vids 2-4094
hwaddress 10:7c:61:4f:27:b0
pointopoint xx.xx.xx.xx
up sysctl -p
post-up ip route add 192.168.2.0/24 via 192.168.1.3
pre-down ip route del 192.168.2.0/24 via 192.168.1.3 || true
post-up ip route add 192.168.20.0/24 via 192.168.1.3
pre-down ip route del 192.168.20.0/24 via 192.168.1.3 || true

iface vmbr0 inet6 static
address 2a01:4f8:120:91ab::2/64
gateway fe80::1

auto vmbr1
iface vmbr1 inet static
address 192.168.3.1/24
bridge-ports none
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
post-up iptables -t nat -A POSTROUTING -s '192.168.3.0/24' -o vmbr0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '192.168.3.0/24' -o vmbr0 -j MASQUERADE
#post-up iptables -t nat -A POSTROUTING -s '192.168.3.0/24' -o vmbr0 -j SNAT --to-source 78.46.102.73
#post-down iptables -t nat -D POSTROUTING -s '192.168.3.0/24' -o vmbr0 -j SNAT --to-source 78.46.102.73

iface vmbr1 inet6 static
address 2a01:4f8:120:91ab:1::1/80

Thanks.

 

[_gabriel]

From the docs :

In some masquerade setups with firewall enabled, conntrack zones might be needed for outgoing connections.
Otherwise the firewall could block outgoing connections since they will prefer the POSTROUTING of the VM bridge (and not MASQUERADE).
Adding these lines in the /etc/network/interfaces can fix this problem:

post-up   iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

https://pve.proxmox.com/wiki/Network_Configuration#sysadmin_network_masquerading

 

[cherimos]

From the docs :

In some masquerade setups with firewall enabled, conntrack zones might be needed for outgoing connections.
Otherwise the firewall could block outgoing connections since they will prefer the POSTROUTING of the VM bridge (and not MASQUERADE).
Adding these lines in the /etc/network/interfaces can fix this problem:

post-up   iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

https://pve.proxmox.com/wiki/Network_Configuration#sysadmin_network_masquerading

This fixes indeed the connection to the internet but at the sametime It lost the connection to the other server which runs at vmbr0.
So when I ping the other vm on the other server it doesnt work anymore. But internal vm does work.
So the solution works for individual environment but not with a cluster.

But I tried nftables and this fixed both my issues I am going to test it for a while but if everything works fine I will post the changes.