[SOLVED] Unexpected Behavior with Multiple Network Interfaces

[ServeTheExperiment]

I have multiple interfaces going into PVE. My server has the following interface built in (Primary: enp87s0) which does not have native drivers built into the kernel. I've been able to install the drivers and get it working on 192.168.1.75/24 (vmbr0 on PVE). For redundancy, I wanted an interface which has native driver support, so I also installed a USB interface (Secondary: enx1cbfceaf8314) on 192.168.1.116 (vmbr1 on PVE). I can toggle the port on the switch side as needed (off if my main interface is working, on if it's not).

/etc/network/interfaces is set up like this:

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface enx1cbfceaf8314 inet manual

iface wlp89s0 inet manual

auto enp87s0
iface enp87s0 inet manual

iface enp88s0 inet manual

auto vmbr0
iface vmbr0 inet static
address 192.168.1.75/24
gateway 192.168.1.200
bridge-ports enp87s0
bridge-stp off
bridge-fd 0

auto vmbr1
iface vmbr1 inet static
address 192.168.1.116/24
bridge-ports enx1cbfceaf8314
bridge-stp off
bridge-fd 0

source /etc/network/interfaces.d/*

I've tested the setup by first testing with both interfaces as operational (192.168.1.75 and 192.168.1.116 both connect). I've removed the cable to my secondary, and the connection works as I expected, but on both IPs, which I did not. Removing the cable to the primary disconnects both IPs and SSH and WebUI become inaccessible.

I suspect this is some bridging error on my part, but how do I ensure that in the event that 192.168.1.75 (primary) is not connected, I can access through 192.168.1.116 (secondary)? Thanks!

 

[gurubert]

The default route (gateway 192.168.1.200) goes via vmbr0, so when you pull the cable from enp87s0 the host is only reachable from within its LAN 192.168.1.0/24.

 

[ServeTheExperiment]

Nevermind. I created an active-backup bond (bond0) between the two, and set that interface (bond0) as the bridge port to vmbr0. It works fine. Thanks for the help. Now I need to make sure, it is using the faster connection when that's available!

 

[gurubert]

Use bond mode active-passive for that with the faster link as active.
If you switch supports LACP you could also use that and utilize both connections.