Traffic issue with Routed setup with multiple IP ranges

[Paspao]

Hello,

I have 2 clusters sharing same switches and some large IP ranges.

All servers have main public IP on 5.x.x.X assigned to bridge

Servers may have 1 or more additional ranges added with routed setup on the bridge, so I can migrate LXCs with different ranges to all nodes of cluster:

up ip route add 45.x.x.0/x dev vmbr0
up ip route add 185.x.x.0/x dev vmbr0
post-up echo 1 > /proc/sys/net/ipv4/ip_forward

I have stp off: bridge_stp off and proxy_arp off

On server A of cluster 1 with routed range 45.x.x.x I noticed traffic on main interface even with VM off and using tcpdump I found it was traffic coming gateway (common to all servers on 5.x.x net) from server B of cluster 2 who has a different routed range 185.x.x.x.

Server A do not have a route for range 185.x .

I even tried to disable ip_forward on server A but I still get traffic from server B.

Do I have to isolate clusters with Vlans to avoid this?

How can improve networking to avoid unneeded traffic but keeping the possibility to use multiple IP ranges on all nodes of the cluster?

Thank you.
P.

 

[Alwin]

While I didn't understand exactly what the issue is, to further isolate the the traffic on the 5.x.x.x network you will need smaller subnets best with VLANs. And a gateway that has filtering abilities to further limit the traffic.