[SOLVED] Two Bridges One Subnet

[ogvirt]

I hope this is a simple answer. I configured a second bridge in the same subnet as my management network (the one created during install). I know this is wrong, but I wanted to gain access to my 10gb NIC and I was trying to cut corners.

Anyways, I'm attempting to fix this, and I want to move vmbr0 to the new subnet. But what is very strange to me is if I physically unplug "the NIC" that vmbr0 is bound to, I lose access to the IP on the second bridge, vmbr1. vmbr1 is bound to the second NIC, which of course is plugged in and responsive - but only when both ports are physically connected. This is so weird to me. This is my /etc/network/interfaces (stripped down to just what should be important, the other stuff just being default)...

auto vmbr0
iface vmbr0 inet static
        address 192.168.40.200/24
        bridge-ports enp6s0
        bridge-stp off
        bridge-fd 0
        mtu 9000

auto vmbr1
iface vmbr1 inet static
        address 192.168.40.201/24
        gateway 192.168.40.1
        bridge-ports enp9s0
        bridge-stp off
        bridge-fd 0
        mtu 9000

Why does .201 stop responding when enp6s0 is unplugged??? enp9s0 is still plugged in and has link. And just to clarify I'm testing from within 192.168.40.0/24 and as far as I'm concerned I could remove the gateway entirely - this isn't a routing issue. Both NICs are completely separate and of course have their own MAC addresses. I'm dealing with an entirely flat network. There must be something very strange about Proxmox networking I'm not grasping at this time?? Thanks!

 

[gurubert]

Because the first ARP request for .201 was answered with the MAC address of vmbr0.
Look into your station's neighbor table to see which IP address currently resolves to which MAC.

 

[lucavornheder]

Yeah. Just like gurubert says, sounds like a arp problem

 

[ogvirt]

Maybe it's an ARP issuse but if it was I'd think I could clear arp table/cache and it would start working, but it did not. And it's still two separate NICs with separate MAC addresses so I don't understand how bridging causes that to get confused.

I was able to get my 2nd bridge/interface to respond by doing ifdown vmbr0 and then I could access the second NIC. But between yesterday and today I've realized I do not know enough to ensure this is working how I want, mainly that my 10gb NIC is being used to connect to my NAS for iSCSI. Until I downed vmbr0 I realized it was not using the 10gb NIC. I've enrolled my network engineer friend to see if he can help me sort this out. It seems I've gone as long as possible running a flat network at home.