Linux Virtual Bridge Issues

[Fazio8]

Hello,
I'm experiencing an issue with multiple LXC containers on Proxmox where only one of two configured network interfaces can access the internet at a time.
Here are the details:

Proxmox Host Configuration
File: /etc/network/interfaces

auto vmbr0
iface vmbr0 inet static
    address 192.168.1.11/24
    gateway 192.168.1.1
    bridge-ports enp3s0f1np1
    bridge-stp off
    bridge-fd 0
#LAN-BR

auto vmbr1
iface vmbr1 inet static
    address 192.168.8.1/24
    bridge-ports none
    bridge-stp off
    bridge-fd 0
#MGMT

auto pir8
iface pir8 inet manual
    bridge-ports none
    bridge-stp off
    bridge-fd 0

auto vmbr3
iface vmbr3 inet static
    address 192.168.201.253/24
    ovs_type OVSBridge

---

LXC Container Configuration
File: /etc/pve/lxc/302.conf
Network interfaces:
net0: name=pir8, bridge=vmbr3, IP=192.168.201.199/24, gateway=192.168.201.1
net1: name=eth0, bridge=vmbr0, IP=192.168.1.33/24, gateway=192.168.1.1

Container Network Configuration
File: /etc/network/interfaces
text
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    address 192.168.1.33/24
    gateway 192.168.1.1

auto pir8
iface pir8 inet static
    address 192.168.201.199/24
    gateway 192.168.201.1

Current Behavior in LXC
eth0 can ping 1.1.1.1 successfully
pir8 cannot ping 1.1.1.1 (100% packet loss)
Both interfaces can ping their respective gateways

Routing Table in LXC:

default via 192.168.1.1 dev eth0 onlink
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.33
192.168.201.0/24 dev pir8 proto kernel scope link src 192.168.201.199

Additional Information
When I rename one of the interfaces on the LXC, the renamed interface gains internet connectivity, but the other loses it. The issue is replicable.
Not DNS related; the problem persists even when just pinging 1.1.1.1.
I've tried various configurations and routing table adjustments, but I can't get both interfaces to have internet access simultaneously.

I face the same issue using classic Linux bridges and OVS bridges. Sometimes, I face connectivity local issues, with LXCs with 2 vmbr (.1.X and .201.X), unable to communicate with each other within the same subnet.

Any assistance or guidance would be greatly appreciated.

 

[Maximiliano]

Hello,

Your container has two gateways, please remove the gateway from the second interface. Perhaps this explain the issue.

 

[Fazio8]

Hello,

Your container has two gateways, please remove the gateway from the second interface. Perhaps this explain the issue.

Why is this a problem?
Both interfaces should exit the network, via gateway. For example, eth0 can reach internet via 192.168.1.1, while pir8 should be able to reach other internal networks.
I do routing via Mikrotik CHR in Proxmox at .1 for all interfaces.
I have no issues with VMs, only LXCs.
What is really anomalous, is that if I manually reload one of the two interfaces (by renaming it to a different name), the newly renamed get connectivity via the expected gateway (if eth0 has connectivity via gateway and pir8 not, I rename pir8 to pir9, pir9 will exit correctly via gateway (192.168.201.1), while eth0 will lose external connectivity until I do the same trick (eth0 renamed in eth1), loosing at this point to the 2nd interface pir9.

 

[FSIN]

Hi,
I had this issue to, removing one Gateway fixed it for me. If i understand correctly in the end all the Traffic goes out of your Mikrotik router, why would you need two gateways in the Container, if the Outgoing Router/Modem is all the same?

 

[Fazio8]

Hi,
I had this issue to, removing one Gateway fixed it for me. If i understand correctly in the end all the Traffic goes out of your Mikrotik router, why would you need two gateways in the Container, if the Outgoing Router/Modem is all the same?

Hi,
My use case:

1. 1 interface has internet access only
2. 1 interface has no internet access, requiring routing to other networks and VLAN.
3. I was also testing the use of 2 different bridges, 1 with standard internet access, 1 with internet access via VPN interface on CHR router.

The point here is that both setup works, until one of the two interfaces stop forwarding traffic to the gateway. I have the same behavior on OVS and classic Linux bridge, so I have no ideas.

 

[FSIN]

I'm kind of confused, but to me it seems to make more sense to use firewall rules for this purpose. 1 you can log the rules and 2 you only have 1 interface in the right VLAN e.g. MGMT or DMZ this makes more sense in terms of network segmentation.

 

[guletz]

Hi,

You must have only one def gw(on the interface with Internet access).
Then on the second interface, you must add static routes on the desired destination thru that vlan.

Good luck / Bafta !

 

[guletz]

@Fazio8 , good ideea with CHR !

 

[Fazio8]

I'm kind of confused, but to me it seems to make more sense to use firewall rules for this purpose. 1 you can log the rules and 2 you only have 1 interface in the right VLAN e.g. MGMT or DMZ this makes more sense in terms of network segmentation.

I know and I understand, however this is still something supposed to work and the behavior is proving an actual issue.