[SOLVED] pfSense in a VM with HA

lifeboy

Active Member
I'm trying to figure out how I could install a pfSense VM and make it HA. I have a ceph cluster of 4 nodes (but there will be nodes added regularly), so I can give each node a public IP address on one of it's ethernet ports. Creating a bridge and adding that to the pfSense VM will give me a WAN port that should be working from any node (same name of each node), depending on the where the Proxmox VM is running. However, pfSense assigns the WAN port to a MAC address. Each node will have a difference MAC address, so depending on which node is running the pfSense VM, the MAC address won't be found, so this is not going to work.

My question is: Will that actually work like I described above? Also, it's there a better way to do this?

The only option I can think of is that I run a pfSense VM on every node and use CARP to sync them. Then if one node goes down, CARP takes care of the failure and nothing is lost.

I'm sure many of you have thought about this or actually done this. What would you propose?
 
Last edited:

Alwin

Proxmox Staff Member
Staff member
Aug 1, 2017
4,617
438
88
Create a bridge, add the NIC port form the public network and add the public interface of the VM. Then assign the IP inside the pfsense. This needs to be the same bridge + NIC port setup on all the nodes. Eg. eth0 -> vmbr0 -> VM | public IP.
 

Jacky Li

Member
Jan 15, 2019
44
2
8
46
Hi,

My pfense runs as a KVM with an virtio nic and a bridge attached to the nic. I then assign different vlans interface for WAN, LAN, DMZ and others in pfsense. I use NFS for my disk storage. I need to migrated it last night and works.
 

lifeboy

Active Member
Hi,

My pfense runs as a KVM with an virtio nic and a bridge attached to the nic. I then assign different vlans interface for WAN, LAN, DMZ and others in pfsense. I use NFS for my disk storage. I need to migrated it last night and works.

All my VM's run on ceph rbd. My objective is to use Proxmox HA feature to automatically migrate the Proxmox VM to a next designated node if the node that it's running on fails. So I can't entertain having to manually change the WAN port parameters once the VM run on the new node. It seems like this is dead end.

A way forward would be to run an instance of pfSense on each node and use CARP as a mechanism to switch to a working node if the running node fails. It's wasteful, since I'll be running a pfSense on each node, but the waste is small.
 

lifeboy

Active Member
pfSense assigns the WAN port based on the MAC address of the port (in this case the bridge). Surely I can't create a bridge on each node with the same MAC address?

Ok, I set up a test machine and it seems that pfSense binds the WAN port to the name of the interface and not the MAC address. The MAC address is part of the KVM part of the bridge, so it stays the same regardless of which node pfSense is running on. So I have two pfSense instances running with CARP doing the failover between then. Furthermore, each instance is also running in an HA config, so if anything goes wrong with a node, pfSense is automatically migrated to another node. If anything goes wrong with pfSense itself, CARP switches to the peer running on another node, so now I have double redundancy for my firewall.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!