Two NICs, two bridges, why does this not work?

[cmt_trts]

Trying to get this to work but I just can't figure out what I'm doing wrong.

Situation is the following: Proxmox server is on a University network and has two ethernet ports (HP Z800 workstation). Both ports are connected to the uni LAN and both MAC-addresses are whitelisted for the network. Any IP on the uni LAN gets internet access.

My setup:
During Proxmox installation, I was asked to enter a static IP, at which point I entered an address that I knew this MAC-address had been assigned previously, and this worked fine. Otherwise the uni network hands out IPs with DHCP for whitelisted MAC-addresses.

Thus, vmbr0 was created automatically, connected to NIC #1 (enp1s0), has an IP and a gateway, and provides the Proxmox server with internet access.
Now, I linked NIC #2 (enp2s0) to a linux bridge (vmbr1) and didn't set any options up, except the bridge port of course.

Then, I created vmbr10 to act as an internal network switch (10.10.10.X-network), set up a pfSense VM which has it's WAN on vmbr1 and LAN on vmbr10. The MAC-address for the network interface connected to vmbr1 is one that is whitelisted on the uni network.

The problem is the following: The pfSense machine doesn't get an IP on it's WAN interface that's connected to vmbr1. There are just no DHCP offers. The internal network on vmbr10 works fine.
I though this was a problem with the uni network not having whitelisted both MAC-addresses, but that's not the case because if I change pfSense's WAN interface to be connected to vmbr0, it get's an IP.
I then thought there was something wrong with the physical connection to the lan from enp2s0, but that turned out not to be the case because if I switch the network cables between enp1s0 and enp2s0, it still works as I described above..

Any ideas?

 

[vesalius]

Show your /etc/network/interfaces file, for better suggestions.

 

[cmt_trts]

auto lo                                                                                   
iface lo inet loopback                                                                     
                                                                                          
iface enp1s0 inet manual                                                                   
                                                                                          
iface enp2s0 inet manual                                                                   
                                                                                          
auto vmbr0                                                                                 
iface vmbr0 inet static                                                                   
        address xxxx.xxx.xxx.xxx/24                                                       
        gateway xxx.xxx.xxx.x                                                             
        bridge-ports enp1s0                                                               
        bridge-stp off                                                                     
        bridge-fd 0                                                                       
                                                                                          
auto vmbr1                                                                                 
iface vmbr1 inet manual                                                                   
        bridge-ports enp2s0                                                               
        bridge-stp off                                                                     
        bridge-fd 0                                                                       
                                                                                          
auto vmbr10                                                                               
iface vmbr10 inet manual                                                                   
        bridge-ports none                                                                 
        bridge-stp off                                                                     
        bridge-fd 0

 

[B.Otto]

So both of your physical ports enp1s0 and enp2s0 are connected to the same switch?

 

[cmt_trts]

So both of your physical ports enp1s0 and enp2s0 are connected to the same switch?

I would assume so, since the ethernet jacks they are connected to are in the same room and have markings that suggest to me that they're just the ports next to each other on the switch. Regardless, any computer connected to these ports end up on the same VLAN, according to the network guys here.
Is it crucial to know if they are actually connected to the same physical switch?

Someone (ok, it was ChatGPT..) suggested me to run tcpdump on enp2s0 and check if there actually are DHCPOFFERS coming on port 68, and there is! Super weird, I can see the both the request from my client connected to enp2s0 and the offer from the DHCP server, but there must be something weird going on where the VM does not actually get the response? Unless I'm misinterpreting the tcpdump response.

I also quickly shut off the pfsense vm and spun up a ubuntu desktop with the whitelisted MAC, connected to vmbr1, and the same thing occurs, so this shouldn't be a pfSense issue I assume?

 

[cmt_trts]

For reference, it appears solved.

Going through the pve-docs for the 16:th time, I found the

bridge-disable-mac-learning-1

option, which I have no idea what it does since I do not understand the explanation in the docs. But at this point, why not just try, nothing makes sense anyway.

After a reboot of the pfsense VM, everything works perfectly. Both physical ports on the host are now used.