L2 Firewall - Traffic disappears across bridge

abbynormal

Member
Oct 18, 2018
4
0
6
31
I have what essentially equates to a L2 FW deployment flow, and I'm seeing some issues with the client not receiving traffic when return L2 FW rewrites the MAC with it's VE MAC. The following is my current setup, where all players exist within Proxmox:

Untitled Diagram.png

I initially had everything on vmbr1, but in my attempts to try and resolve possible issues, I created vmbr3. The only place vmbr3 exists on Proxmox is on the two vNICs above, for the LXC and the FW KVM. So the link between the LXC and FW should be completely L2 segmented.

And to be clear, the breaking point is specifically on this link, between LXC and the FW KVM:

Untitled Diagram1.png

LXC vNIC config:
lxc_network.png

FW VM vNIC config:
fw_network.png

I have taken three simultaneous captures to try and correlate the issue:

  1. Capture on FW
  2. tcpdump within LXC
  3. tcpdump directly in Proxmox host, against vmbr3

I've noticed that traffic is forwarded fine unless FW rewrites source MAC to it's VE MAC for traffic from Proxy to LXC. In the captures we see this traffic on FW, and vmbr3, but never on the client. We also never see the retransmissions from the client after the first connect request:


LXC:
1602980917035.png


FW VM:
1602981263151.png


vmbr3:
1602981181189.png




This flow defiantly has me scratching my head on what the heck is going on.

  • From the tcpdump against the vmbr3, it seems like we see traffic in both directions with the full expected MACs
  • From the client, we never seen the return traffic when the source MAC is the FW VE interface
  • From the FW, we see the initial "connect" request, however we never see any of the follow up retransmissions for the client


Any idea what would cause this behavior? My suspicion is some kind of "gotchas" in the Proxmox vSwitch for such a flow that I'm unaware about.
 
Last edited:

abbynormal

Member
Oct 18, 2018
4
0
6
31
SOLVED:
Seems issue was indeed a direct result of the vswitch behavior. Above all references to vbridges were with the standard Linux bridges. As a test, I created two new bridges, each tied to one unused phyiscial NIC. Then I cabled both of these NICs together. Finally, I tied one vbridge to the client vNIC and the other to the FW vNIC.

In doing so, I saw the exact same behavior. IE client-side vbridge capture matched up with above LXC capture, and FW side vbride capture linedup with above capture taken in FW. Seems something within Linux bridge behavior does not play nicely with this L2 flow.

I installed OVS and made an OVS bridge to tie the client and FW together. Worked right away....
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!