blocking traffic in-between VMs

[Stefano Giunchi]

Hi,
I have a pve on Internet, I want to block any traffic between VMs, and allow them to go to Internet only.

I enabled the firewall on datacenter, node and vm level.

The node firewall works, I can only connect to it from my office public IP address, but the VM pve firewall doesn't DROP anything.

The default IN rule in all VMs firewall is DROP, anyway if from VM with IP 10.10.10.100 I do a "curl -k https://10.10.10.101", I get an answer from the webserver.

The firewall configurations only contain the "enable: 1" flag, there isn't the default DROP rule, probably because it's the default.

Thank you for any help.

EDIT: I add this is a nested PVE.

 

[Stefano Giunchi]

The node firewall works, I can only connect to it from my office public IP address, but the VM pve firewall doesn't DROP anything.

I'm looking again now to the firewall, and I see now that the firewall is completely open, not blocking anything for both the nodes and the VMs.
Probably I was wrong when I created the post, or something changed.

I tried restarting pve-firewall and dis/enable the firewall settings, without success.
I still haven't tried to reboot the server.

These are my config files:

~# cat /etc/pve/firewall/cluster.fw
[OPTIONS]

enable: 1
policy_in: DROP

[IPSET lan10]
10.10.10.0/24

[IPSET management]
88.88.88.88/29 # Office Public IP #1
77.77.77.77/29 # Office Public IP #2

[RULES]
IN ACCEPT -source +management -log nolog

:~# cat /etc/pve/nodes/pve1/host.fw
[OPTIONS]

log_level_in: info
enable: 1

And this is my running iptables rules (I'm no expert, but it seems everything open to me)

~# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
f2b-sshd   tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain f2b-sshd (1 references)
target     prot opt source               destination
REJECT     all  --  123.123.123.123        0.0.0.0/0            reject-with icmp-port-unreachable
REJECT     all  --  234.234.234.234        0.0.0.0/0            reject-with icmp-port-unreachable
[...]
RETURN     all  --  0.0.0.0/0            0.0.0.0/0

I tried this:

iptables -P INPUT DROP

And I effectively kicked me out.
Thus iptables works, but it seems to me that the pve-firewall rules are not transformed in iptables rules.

 

[Stefano Giunchi]

I found this error in /var/log/syslog:

/var/log# pve-firewall restart
Dec 30 17:02:36 pve1 systemd[1]: Reloading Proxmox VE firewall.
Dec 30 17:02:36 pve1 pve-firewall[2448715]: send HUP to 1278
Dec 30 17:02:36 pve1 pve-firewall[1278]: received signal HUP
Dec 30 17:02:36 pve1 pve-firewall[1278]: server shutdown (restart)
Dec 30 17:02:36 pve1 systemd[1]: Reloaded Proxmox VE firewall.
Dec 30 17:02:37 pve1 pve-firewall[1278]: restarting server
Dec 30 17:02:37 pve1 pve-firewall[1278]: status update error: ipset_restore_cmdlist: Try `ipset help' for more information.

Please note: this is a PVE7.1 with old kernel. There's a reason for this, otherwise I must go back to PVE6 rebuilding everything.

Some more informations:

:/var/log# pveversion -v
proxmox-ve: 7.1-1 (running kernel: 5.4.143-1-pve)
pve-manager: 7.1-7 (running version: 7.1-7/df5740ad)
pve-kernel-5.15: 7.1-6
pve-kernel-helper: 7.1-6
pve-kernel-5.13: 7.1-5
pve-kernel-5.4: 6.4-7
pve-kernel-5.15.5-1-pve: 5.15.5-1
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.4.143-1-pve: 5.4.143-1
ceph-fuse: 14.2.21-1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-5
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-14
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.0-4
libpve-storage-perl: 7.0-15
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.9-4
lxcfs: 4.0.8-pve2
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.1.2-1
proxmox-backup-file-restore: 2.1.2-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-4
pve-cluster: 7.1-2
pve-container: 4.1-2
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-3
pve-ha-manager: 3.3-1
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.0-3
pve-xtermjs: 4.12.0-1
qemu-server: 7.1-4
smartmontools: 7.2-pve2
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.1-pve3

# uname -a
Linux hiddenname.contaboserver.net 5.4.143-1-pve #1 SMP PVE 5.4.143-1 (Tue, 28 Sep 2021 09:10:37 +0200) x86_64 GNU/Linux

~# pve-firewall status
Status: enabled/running (pending changes)

 

[Stefano Giunchi]

The main firewall problem on the node seems resolved using this patch.
Ref: https://forum.proxmox.com/threads/firewall-stuck-on-pending-changes.98418/

EDIT: anyway, the vm/ct firewall still doesn't work