Route all VMs through a firewall VM - confused about bridges

[TomSawyer]

I want all my VMs to go through my firewall VM. I know I need to bridge all of them but when I check the network on the host it shows the pve IP I chose during install.

Do I need to add a new Linux Bridge for every VM or just use the same vmbr0 for all of them and proxmox will do the rest and route/switch between them?

The only thing that confuses me is the pve's IP assignment on that vmbr0 bridge.

I'm thinking of this solution - leave vmbr0 (pve's bridge) untouched, add a new bridge vmbr1 and make all the VMs use it as a "switch", assign this new bridge as a network device on the pfSense VM and I think it should work, but then how do I also bridge the pve host through pfsense? Add its bridge as a network device too in the pfSense VM?

My network looks like this:


Thank you,

 

[sherminator]

Hey!

to put all VMs "behind" your pfSense VM I would do what you're suggesting:
Leave the "default bridge" vmbr0 untouched and create a second one "vmbr1". Give two vNICs to your pfSense VM, assign the "WAN" interface to vmbr0 and the "LAN" interface to vmbr1. Finally connect all the other VMs to vmbr1.
I have no idea whether it's possible to put a PVE host "behind" a VM, but anyhow I wouldn't do it. Call me "old school", but I strongly feel that any setup in which the PVE host is dependent on a VM running inside, is not so cool.
For me putting a firewall in front of a PVE host means putting a second piece of hardware in front of it.

Greets
Stephan

 

[TomSawyer]

but I strongly feel that any setup in which the PVE host is dependent on a VM running inside, is not so cool.

I understand your point and the scenarios of not having remote access if pfSense goes down and I too feel that the firewall should be a standalone bare metal equipment but currently I don't have spare parts for something so minimal as firewall (in terms of hardware) and I don't wanna waste killer hardware for that.

That being said, how else would the pve get DHCP and Internet connection if not from the firewall VM which suppose to manage all the network?

Edit: I've tried to add vmbr0 to pfsense as an interface (NIC) but the only thing it did was preventing me from accessing the pve host and bridging whatever I connected to that physical port (the only one I have on the motherboard) to the pfSense VM for DHCP etc.
So I guess the solution for routing the pve host itself through a firewall VM is to physically connect its port to an external switch controlled by the pfsense VM. It would be a weird circle of connection but will achieve what I want. For now I don't want to connect the pve to a physical gateway and I don't have spare parts. That's one of the reasons why I'm virtualizing my firewall - because almost every hardware dedicated for it would be a waste of hardware, until there would be a small, very cheap, very minimal hardware for firewalls.

See the image below:

 

[sherminator]

My best idea so far is:
1. do a PCI passthrough of your physical NIC into the pfSense VM (as "WAN" interface).
2. connect the second vNIC of this VM to vmbr0 ("LAN" interface)
3. connect your PVE host and all the other VMs to vmbr0

Still sounds wrong to me :-D But maybe somebody can say whether it's technically possible.

Greets
Stephan

 

[TomSawyer]

I already got the behavior I wanted but with one extra cable. I'll change it later to be completely internal (except the PCI-E passthrough of course).

Currently I'm passing an Intel PCI-E card to the FW VM and connecting the host with a cable to a switch that is controlled by the FW VM via that PCI-E card. It's sort of a loop haha but it works.