3 NICS 2 bonded with 802.3ad plus failover to second switch?

[jacs]

Hi,

Ive got two Nic's on each of my Proxmox servers aggregate bonded together via 802.3ad to my vmbr and connected to a high speed switch. I have a spare third Nic on each server which I would like to connect to a lower speed switch as a failover. Just incase the high speed switch fails or is rebooted. Is this possible? I don't have the ability to 802.3ad across switches.

Thanks

Chris

 

[weehooey-bh]

What kind of failover are you looking for? PVE node management? Corosync? Guest traffic?

There are several different scenarios for failover.

More information about your current network configuration and the failure you are trying to mitigate would be helpful.

 

[jacs]

What kind of failover are you looking for? PVE node management? Corosync? Guest traffic?

There are several different scenarios for failover.

More information about your current network configuration and the failure you are trying to mitigate would be helpful.

Thanks for replying. What I am worried about is having HA switched on with Ceph and the Ceph getting corrupted with a network outage. This was a major issue when I used VMWare where if the vsan got corrupted due to total network outage it was a real pain sorting it out.

 

[weehooey-bh]

Thanks for the additional information.

What I am worried about is having HA switched on with Ceph and the Ceph getting corrupted with a network outage

Ceph is remarkably resilient. I have not seen it get corrupted. I am not saying it cannot; I have not seen it.

If you can, get redundant switches for your high-speed links. There are two approaches you can take.

#1 Best

Two switches that support MLAG. You still use 802.3ad on the bond, and the switches would be present as one connection. You get the full bandwidth of each; if one fails, you will still have 50% of your bandwidth. The switches need to support MLAG.

Depending on your switch vendor, it may have different names. Cisco has many names for it. Cisco Nexus calls it Virtual Port Channel (vPC), and Juniper calls it multi-chassis LAG.

#2 Half As Good

You do not need the switches to support this approach. You set the bond in active-backup and connect one link to each switch. You lose half your bandwidth, but if a switch fails, the connection will failover.

Plan for Extra NIC

Since either of the approaches above will allow you to achieve the resiliency you seek, you can still use the extra NIC. Make it your primary or backup Corosync link on a dedicated switch, assuming the NIC is at least 1 Gbps.