[SOLVED] pve-firewall drop traffic

Nikolay

New Member
Jul 16, 2016
16
1
3
I have strange network issues. There are 2 physical servers, PVE1, PVE2. Both have virtual routers with 2 dedicated network interface, shared with bridge utils. 2 different ISP connected to R1 and R2. Branch office connected to both routers with l2tp tunnel.

R1, have default gateway to network 192.168.236.1/24, R2 have 192.168.236.2/24.

If traffic comes from branch through R2 it doesnt return from VMs, located at PVE2. At PVE1 traffic come back through R1 or R2 as normal. I can see with tcpdump traffic at VM5, located at PVE2. But when VM5 send icmp reply, as example, to R1, it doesnt reach tap501i1 interface, but i can see this reply at vmbr1 with tcpdump.

When VM5 send echo request to host at branch office, it gets reply. But it cant make TCP sessions, coz TCP syn doesnt come.

It looks like firewall issues, but i bound firewall rules to vmbr0 only, that looks to internet. I have tried to disable firewall with no luck.


proxmox-ve: 4.4-76 (running kernel: 4.4.35-1-pve) pve-manager: 4.4-1 (running version: 4.4-1/eb2d6f1e) pve-kernel-4.4.35-1-pve: 4.4.35-76 pve-kernel-4.4.21-1-pve: 4.4.21-71 lvm2: 2.02.116-pve3 corosync-pve: 2.4.0-1 libqb0: 1.0-1 pve-cluster: 4.0-48 qemu-server: 4.0-101 pve-firmware: 1.1-10 libpve-common-perl: 4.0-83 libpve-access-control: 4.0-19 libpve-storage-perl: 4.0-70 pve-libspice-server1: 0.12.8-1 vncterm: 1.2-1 pve-docs: 4.4-1 pve-qemu-kvm: 2.7.0-9 pve-container: 1.0-88 pve-firewall: 2.0-33 pve-ha-manager: 1.0-38 ksm-control-daemon: 1.2-1 glusterfs-client: 3.5.2-2+deb8u2 lxc-pve: 2.0.6-2 lxcfs: 2.0.5-pve1 criu: 1.6.0-1 novnc-pve: 0.5-8 smartmontools: 6.5+svn4324-1~pve80 openvswitch-switch: 2.6.0-2
 
Confirmed, it`s firewall issue. When i disabled firewall globally, everything works fine.

rp_filter=0 at all interfaces, firewall rules applyed only to vmbr0 - it`s my external network. vmbr1 doesnt have anykind of rules applied.

Any suggestions?
 
Selection_043.png


how to reproduce bug:

1) Use assymetric routing
2) Enable pve-firewall with default policy accept, u dont need any kind of rules.
3) Get traffic drop at R1.
 
god damn, bridge-nf-call-iptables...


When i enable PVE-Firewall, it uses net.bridge.bridge-nf-call-iptables=1.

So in my case, solution - disable pve-firewall and never use it.