2x Opnsense CARP HA VMs in one host

L

listhor

Member
Nov 14, 2023
49
2
13
I’m trying to configure 2x Opnsense instances in high availability mode with carp vip interfaces on single pve host. I know it’s not full HA but I want software HA and also simply to test it.
VMs are connected through 2 bridges: 1 on WAN side, the other on LAN side (and further trunk physical link to switch).
IGMP snooping is disabled in (UniFi) switches.

But problem I’m facing is duplicated communication/data flow to and from both VMs; both instances have same looking graphs in webgui - network flow and also cpu. Despite they don’t change their master/backup status (no flapping at carp status) I have something similar to split brain situation, for example if I communicate with webgui or ssh on carp vip interface, reply comes either from one of those two and toggles every few seconds. If I ping them, reply is duplicated (“DUP!”). Communication to other hosts and WAN is ok.

I have already set Mac filter to “no” in VM’s firewall options (pve firewall is disabled). I tried ovs and Linux bridges with same results.
Is it possible to set it up correctly?
 
did you configure a "proxmox internal" isolated bridge for the private communication between the opesense (IIRC it's named pfsync in opensense doc) ?
 
Etienne Charlier said:
did you configure a "proxmox internal" isolated bridge for the private communication between the opesense (IIRC it's named pfsync in opensense doc) ?
Click to expand...
Yes, I forgot to mention that. There’s third bridge purely for pfsync. But issue (traffic duplication) is related to lan and wan bridges
 
What if I create additional LAN bridge for backup instance and instead of having them (2x opnsense) connected over single linux bridge - within proxmox, connect them over physical switch?
This of course requires second downlink:
  • master/regular bridge would remain connected as it is now
  • backup/new bridge will be connected to switch via additional downlink
Few running LXC containers would stay connected to master bridge. Would this work?
 
Anybody, anything?

Changing setup by adding separate LAN bridge with its downlink didn't help.
Is there anything else I could try?
 
You must log in or register to reply here.
Top Bottom
Back