Proxmox VMs on the same network adapter ignoring local firewall rules

[bracks917]

Hi,

As per title, am having issues with virtual machines within proxmox, using the same network adapter / vmbr completely bypass the firewall rules in place on the router.
I guess Proxmox is doing local switching, and so requests from other VMs never leave the proxmox server and circumvent the local firewall rules.

Is there an easy way to disable this, without having to maintain a second set of firewall rules for each proxmox?
Would the super jank option of a network adapter for every vm work?

Thanks

 

[shrdlicka]

Hi,

I don't exactly understand the situation, can you post your /etc/network/interfaces config and show one of the configs of the problematic vms

qm config <vmid>

 

[bracks917]

Hi,

I don't exactly understand the situation, can you post your /etc/network/interfaces config and show one of the configs of the problematic vms

qm config <vmid>

Hi,

Sorry to give a bit more detail, we have pretty strict firewall rules. One of the VLANs in use hosts cant communicate with other hosts on the same subnet, they basically just have internet access.

We have about 5 proxmox servers, and for example if we have 4 VMs on the same proxmox server, that are using the same network interface (vmbr0 for example) that is on the strict VLAN those VMs are able to SSH to each other (every port is open). From what i can gather the proxmox server isn't forwarding the requests to the router it is instead acting as a local switch and allowing the VMs to directly communicate bypassing our firewall rules.

Basicalling I'm wondering is there a way to stop this from happening without having to maintain a duplicate set of firewall rules on each proxmox server we have. I don't want to make a firewall change 6+ times every time one is required.

boot: order=scsi0;ide2;net0
cores: 9
ide2: local:iso/ubuntu-20.04.4-live-server-amd64.iso,media=cdrom
memory: 81920
meta: creation-qemu=6.1.0,ctime=1648030895
name: *****-1
net0: virtio=2E:5A:3E:AB:C7:5F,bridge=vmbr1,firewall=1
numa: 0
ostype: l26
scsi0: local-lvm:vm-101-disk-0,size=2T
scsihw: virtio-scsi-pci
smbios1: uuid=bceb2fc7-4f50-467f-be46-74d1d7bd1d83
sockets: 2
vmgenid: 98c450a8-4264-4364-870a-de8bd271f283

auto lo
iface lo inet loopback

auto enp5s0f1
iface enp5s0f1 inet manual

auto eno1
iface eno1 inet manual

auto eno2
iface eno2 inet manual

auto enp5s0f0
iface enp5s0f0 inet manual

auto eno3
iface eno3 inet manual

auto eno4
iface eno4 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno1 eno2
        bond-miimon 100
        bond-mode 802.3ad
        bond-xmit-hash-policy layer2

auto bond1
iface bond1 inet static
        address *.*.*.*/24
        gateway *.*.*.*
        bond-slaves enp5s0f0 enp5s0f1
        bond-miimon 100
        bond-mode active-backup
        bond-primary enp5s0f0

auto vmbr0
iface vmbr0 inet manual
        bridge-ports bond1.9
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet manual
        bridge-ports bond1.10
        bridge-stp off
        bridge-fd 0

auto vmbr2
iface vmbr2 inet static
        address *.*.*.*/24
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0

 

[shrdlicka]

Lets see if I understood you correctly:
There are VMs on your host that have on one of the bridged interfaces (for example vmbr0). All the host exist in the same VLAN and within the same IP subnet.

Is this correct?

 

[bracks917]

Lets see if I understood you correctly:
There are VMs on your host that have on one of the bridged interfaces (for example vmbr0). All the host exist in the same VLAN and within the same IP subnet.

Is this correct?

Yes, the switch has a defined vlan 10.x.x.x/24 vlan id 9. and associated firewall rules to go with it blocking communication between other hosts on the same vlan.

In proxmox vmbr0 is configured using that vlan id and there are 4-5 VMs within proxmox using that network adapter. Proxmox is doing local switching so instead of VM > switch (with firewall rules) > VM Proxmox is not forwarding the packets and allowing the VMs to communicate directly.

Found two solutions, neither of which are ideal. Setting individual firewall rules for each VM.
This is a flat no, 32 host with firewall rules to update is too much

Option two is to set the subnet in netplan to /32 forcing the firewall rules to be enforced, however this is done on the host OS and can be circumvented.

I'm wondering if there is an option within proxmox to disable the local switching or set the firewall rules in one place. I have tested in the datacenter firewall rules, the proxmox server firewall rules and the VM firewall rules. The only place the firewall rules work is in the VM settings, and this isn't feasible.

 

[shrdlicka]

If the VMs do not need to talk to each other at all you can put each VM in its own VLAN by setting it in the VMs network device settings. But I'm not sure if this just moves the configuration burden to some other location.

 

[shrdlicka]

Oh btw are you familiar with the security groups? This would make setting the rules for containers easier, which you can then role out for all containers with some simple bash copy&paste.