pfSense HA setup with 4x node cluster

[gowger]

I'm hoping this will help some others out there, but also would like to assess the viability of this solution.

I will get a /29 ip address block with 5 useable IP addresses from the colocation data centre where it will be hosted.

I should be able to pre-configure off site the pfsense firewall/router, set one of those external IP addresses as a static IP, set it's gateway, configure it to do NAT to local network addresses for IPMI nics, cluster nics, and vms.

I don't know exactly what, if any switch config might need doing to support this setup. The switch is a Cisco WS-C4948-S 48 Port. Does it need to have VLANs configured before any of this will work?

Ideally all external traffic goes through pfSense so nothing is exposed to internet directly. As long as at least one server node is up, a pfsense instance should be up and this will be possible to get access to IPMI etc if necessary.

I believe I need to keep one NIC dedicated to the pfSense WAN side on every host. Which means that I'll only have 1Gbs bandwidth for internal cluster traffic, not ideal. Perhaps there is a better way?

 

[wolfgang]

Hi,

when I understand you correctly you have 4 ports on each cluster node?

Do you have a private network on the switch?
On what network do you create the cluster?

 

[gowger]

Thanks for the reponse. I have 2x 1GbE ports per server node currently, and an IPMI port also.

I would be intending to have the cluster on the private network, and expose only via pfSense. I have no experience with cisco rack switches, so I don't know what would be involved to set that up for both the public and private LAN, or if anything even actually needs to be done at all.

This switch also has 4 10Gb uplinks, but I don't know if these can be used for the LAN.

 

[wolfgang]

I don't know how you setup works but I would work with VLAN's and isolate your Uplink in a separate Network.
Then tag the WAN interface of the pfsence with this VLAN and the LAN network is use untagged.

Also, use one nic exclusive for corosync traffic.
This ensures the network is not disturbed.

 

[gowger]

Much appreciated, and appreciate the point about corosync traffic also!