[SOLVED] PVE 6.3-3 Firewall: block traffic between VMs on same node

[Asg.Systems]

Dears,
We have the following scenario:

A Proxmox Cluster with version 6.3-3, kernel version 5.4.78-2 with 2 nodes:

On network side we're using linux bridge with vlan awareness and configuration of one node is the following:

auto lo
iface lo inet loopback

auto eno5
iface eno5 inet manual
        mtu 9000

auto eno6
iface eno6 inet manual
        mtu 9000

auto bond0
iface bond0 inet manual
        bond-slaves eno1 eno3
        bond-miimon 100
        bond-mode active-backup
        bond-primary eno1
#MGMT

auto bond1
iface bond1 inet manual
        bond-slaves eno5 eno6
        bond-miimon 100
        bond-mode 802.3ad
        bond-xmit-hash-policy layer2+3
        mtu 9000
#VM-NET

auto vmbr0
iface vmbr0 inet static
        address 10.19.64.103/24
        gateway 10.19.64.254
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
#MGMT

auto vmbr1
iface vmbr1 inet manual
        bridge-ports bond1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
        mtu 9000
#VM-NET

On this node we have the following VMs:

- 1 Cisco ASAv 9.12.3-12 with interfaces one in vlan 101 and one in vlan 100
- 1 VM CENTOS8 on vlan 101
- 1 Container Ubuntu 20.04 on vlan 100

WIth Datacenter Firewall disabled, everythings works fine, when we enable the Firewall, with the configuration like below:

[OPTIONS]

ebtables: 1
policy_in: ACCEPT
policy_out: ACCEPT
enable: 1

on the Node and the VMs the Firewall is disabled.

The TCP traffic stop working:

- ICMP OK
- UDP OK
- TCP broken


looking in tcpdump we see the following:

- on ASA interfaces, we see all packets, TCP SYN and TCP SYN ACK
- on client Interface, we see only TCP SYN and never the SYN ACK

seems to be that the SYN ACK is never replied to the VM, when we disable the firewall at Datacenter level, everythings work again.

Best Regards,
Thank you in advantage

 

[Asg.Systems]

I have some news, I reduced the focus on the ASA, and to be more specific on the ASA TCP sequence randomization function.
Disabling it in this way
https://www.cisco.com/c/en/us/td/do...config/conns-connlimits.html#ID-2068-000003ec

all works, now I want to discover why iptables block the SYN packets, and which rules matches

 

[Asg.Systems]

Solved, to avoid this without disabling the feature on the ASA or the Firewall on Proxmox it's possible to enable this feature:

nf_conntrack_allow_invalid: 1

in /etc/pve/nodes/<nodename>/host.fw