Firewall rules for forward chain

[padelt]

Hi, I can secure my Proxmox 5.0 host with the new (for me) firewall feature just fine - great!

What I am missing now is a way to also protect my VMs that are on a bridge (Hetzner additional subnet setup) with the host having one of the subnet IPs.

AFAIK, packets coming in for a VM are filtered through the FORWARD chain of iptables as they are indeed forwarded by the host into the bridge.

Is there a way to configure rules in that chain using the UI? If not, is there a non-harmful (and reliable) way to add rules manually (e.g. via a on-boot script)?

Philipp

 

[dietmar]

What exactly is the problem? Proxmox VE firewall should work out of the box with VMs.

 

[padelt]

So this is the configuration on the host:

root@h128:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
...
2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
link/ether 38:XX:XX:XX:42:f5 brd ff:ff:ff:ff:ff:ff
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 38:XX:XX:XX:42:f5 brd ff:ff:ff:ff:ff:ff
inet 78.XX.XX.164/27 brd 78.XX.XX.191 scope global vmbr0
valid_lft forever preferred_lft forever
4: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 82:XX:XX:XX:XX:ae brd ff:ff:ff:ff:ff:ff
inet 46.XXX.XXX.1/28 brd 46.XXX.XXX.15 scope global vmbr1
valid_lft forever preferred_lft forever​

root@h128:~# arp -an
...
? (46.XXX.XXX.8) at 6a:XX:XX:XX:2c:d5 [ether] on vmbr1​

When I run a service on 46.XXX.XXX.8 that listens to a TCP port (let's say "nc -l -p 12345"), I can reach that port from the internet (e.g. "echo test | nc 46.XXX.XXX.8 12345" shows up on the destination).

Now I setup a firewall rule on the host that yields

root@h128:~# grep \.8 /etc/pve/local/host.fw
IN REJECT -dest 46.XXX.XXX.8 -p tcp -dport 12345​

and I'd expect that my service is no longer reachable from the internet.

But it still is.

Inspecting the iptables rules on the host, this makes sense to me:

root@h128:~# iptables -L -n -v
Chain INPUT (policy ACCEPT 53 packets, 3080 bytes)
pkts bytes target prot opt in out source destination
77M 62G PVEFW-INPUT all -- * * 0.0.0.0/0 0.0.0.0/0

Chain FORWARD (policy ACCEPT 11265 packets, 576K bytes)
pkts bytes target prot opt in out source destination
192M 148G PVEFW-FORWARD all -- * * 0.0.0.0/0 0.0.0.0/0

Chain PVEFW-FORWARD (1 references)
pkts bytes target prot opt in out source destination
200K 9964K DROP all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate INVALID
190M 148G ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 PVEFW-FWBR-IN all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in fwln+ --physdev-is-bridged
0 0 PVEFW-FWBR-OUT all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out fwln+ --physdev-is-bridged
1279K 85M all -- * * 0.0.0.0/0 0.0.0.0/0 /* PVESIG:xx */

Chain PVEFW-FWBR-IN (1 references)
pkts bytes target prot opt in out source destination
0 0 PVEFW-smurfs all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate INVALID,NEW
0 0 all -- * * 0.0.0.0/0 0.0.0.0/0 /* PVESIG:xx */

Chain PVEFW-FWBR-OUT (1 references)
pkts bytes target prot opt in out source destination
0 0 all -- * * 0.0.0.0/0 0.0.0.0/0 /* PVESIG:xx */

Chain PVEFW-HOST-IN (1 references)
pkts bytes target prot opt in out source destination
3 156 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
3 120 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate INVALID
1833 431K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
717 31303 PVEFW-smurfs all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate INVALID,NEW
0 0 RETURN 2 -- * * 0.0.0.0/0 0.0.0.0/0
0 0 PVEFW-reject tcp -- * * 0.0.0.0/0 46.XXX.XXX.8 tcp dpt:12345

Chain PVEFW-INPUT (1 references)
pkts bytes target prot opt in out source destination
77M 62G PVEFW-HOST-IN all -- * * 0.0.0.0/0 0.0.0.0/0
14264 866K all -- * * 0.0.0.0/0 0.0.0.0/0 /* PVESIG:xx */​

The rule ends up in the path for chain INPUT, but not for chain FORWARD.
AFAIK, packets from the internet going to the bridge never see the INPUT chain (or OUTPUT on the way back) but only the FORWARD chain.

My gut reaction was "hmm, the 'direction' dropdown is missing 'forward'".

Philipp

 

[Richard]

When I run a service on 46.XXX.XXX.8 that listens to a TCP port (let's say "nc -l -p 12345"), I can reach that port from the internet (e.g. "echo test | nc 46.XXX.XXX.8 12345" shows up on the destination).

Now I setup a firewall rule on the host that yields

root@h128:~# grep \.8 /etc/pve/local/host.fw
IN REJECT -dest 46.XXX.XXX.8 -p tcp -dport 12345​

and I'd expect that my service is no longer reachable from the internet.

But it still is.

AFAIU the service runs in a guest, not in the host. Therefore you have to define a rule for the guest - but the following you wrote says it exactly:

AFAIK, packets from the internet going to the bridge never see the INPUT chain (or OUTPUT on the way back) but only the FORWARD chain.

However, you can:

- add rules for the respective VM and they will be transfered to settings in a subchain of the FORWAD chain properly

- add own rules directly into the FORWARD chain - these rules will not be affected when [de/]activating firewall-settings by Proxmox GUI; note: Proxmox created rules will always be appended to those.

 

[padelt]

AFAIU the service runs in a guest, not in the host. Therefore you have to define a rule for the guest

OK so that got me thinking - and I realized my basic mistake: I defined the firewall rules for the host (ending up in /etc/pve/firewall/cluster.fw). I didn't realize the VMs have their own firewall settings.

- add rules for the respective VM and they will be transfered to settings in a subchain of the FORWAD chain properly

OK so that is what I am looking for!

Turns out I missed another gem in the manual (https://pve.proxmox.com/pve-docs/chapter-pve-firewall.html#pve_firewall_vm_container_configuration):

Each virtual network device has its own firewall enable flag. So you can selectively enable the firewall for each interface. This is required in addition to the general firewall enable option.​

So what I was missing in addition was to set the checkmark in Datacenter -> host -> VM -> Hardware -> Network Device.
After that, the firewall rule appeard in iptables and the port was actually firewalled - yeah!

The manual also says this:

The firewall requires a special network device setup, so you need to restart the VM/container after enabling the firewall on a network interface.​

That does seem to be wrong. I did not need to restart my qemu VM for the firewall on the network device to be enabled and effective.

RTFM would've helped, sorry!
That said: As much as I love the GUI, it lures even an old iptables user like me into a lot of entangled footguns.

Thanks for your help!