Proxmox & isp modem without a router

[Gavcol]

Hi all,

The plan was to replace my consumer router with a pfsense box which went well until I made a change and broke the internet, unrecoverable (for me).
Lucky I still had the old router lying around to plug and play and happy kids/wife/life... for now

Then came the suggestions for a pfsense vm on proxmox, so with some early playing around with proxmox and I really really like it.

I had pfsense on a mini pc (i7 6port Protectli/Qotom like box). It's so overkill for pfsense on its own that it's perfect for a hypervisor running pfsense in a vm.
I'm a complete newb to proxmox (though I played around with virtualbox a few yrs ago), I'm having trouble trying to get proxmox configured for pfsense when using only the isp modem (remember I want to replace my Asus merlin router and just have pfsense running the show.

I appreciate this is not a pfsense board and I'm only asking for proxmox help with the network config to pass/bridge the WAN connection from the modem (while not being exposed to the internet itself) and pass the WAN through to pfsense. (From there I'll configure pfsense with firewall rules, VLANs etc.)

Then I'd like to connect the proxmox box to the pfsense vm for it's internet gateway (mostly for updates).
i.e. ISP modem -> proxmox box (6port mini pc server) -> pfsense vm (then provide internet back to proxmox) but I'm having trouble with the bridging etc in proxmox.
Is this even possible ?

I've trawled through a heap of blogs, forums, guides and videos but couldn't find anyone that's detailed how to do this or a proxmox config without another router between the modem and the proxmox box.

This is an idea of most of what I had with pfsense on its own but now I'm trying to put pfsense on a vm and I'm tearing my hair out with the bridging setup.
Any thoughts or advice are welcome and appreciated.

Thanks in advance
Gav

 

[leesteken]

In principle, if you create a virtual bride for each network port, you can pass them all to a router VM (pfSense or other router OS).
(Or you can try PCI passthrough, but I don't think you will gain much and Proxmox will be unreachable when it goes down.)

Install Proxmox, create 5 additional bridges with one physical port each and create a router VM with 6 para-virtualized network devices.
Then configure your router OS from an external system via one of those ports. Be careful not to lock yourself out of the router and/or Proxmox.
Proxmox only needs a single IP address on one of those virtual bridges to be reachable.
You can connect your modem to the bridge/port that you choose to be your WAN connection. Maybe DHCP to the ISP might give you a bit of trouble to configure.

 

[Gavcol]

In principle, if you create a virtual bride for each network port, you can pass them all to a router VM (pfSense or other router OS).
(Or you can try PCI passthrough, but I don't think you will gain much and Proxmox will be unreachable when it goes down.)

Install Proxmox, create 5 additional bridges with one physical port each and create a router VM with 6 para-virtualized network devices.
Then configure your router OS from an external system via one of those ports. Be careful not to lock yourself out of the router and/or Proxmox.
Proxmox only needs a single IP address on one of those virtual bridges to be reachable.
You can connect your modem to the bridge/port that you choose to be your WAN connection. Maybe DHCP to the ISP might give you a bit of trouble to configure.

@avw awesome, thank you. I think I understand and will give it a try.

 

[bobmc]

I've got pretty much the same setup running in several locations (business and home) - works really well. The only caveat is that internet access is wholly dependant on proxmox and pfsense being up, so you won't be popular at home if you need to do any maintenance on your proxmox host

There's a very good guide on the netgate wiki specifically on proxmox/pfsense setup

https://docs.netgate.com/pfsense/en/latest/recipes/virtualize-proxmox-ve.html

My main piece of advice is don't try and get everything setup all at once, get proxmox routing traffic for lan clients first, then make sure you can route vm traffic, then start to look at VLAN, etc

 

[Gavcol]

I've got pretty much the same setup running in several locations (business and home) - works really well. The only caveat is that internet access is wholly dependant on proxmox and pfsense being up, so you won't be popular at home if you need to do any maintenance on your proxmox host

There's a very good guide on the netgate wiki specifically on proxmox/pfsense setup

https://docs.netgate.com/pfsense/en/latest/recipes/virtualize-proxmox-ve.html

My main piece of advice is don't try and get everything setup all at once, get proxmox routing traffic for lan clients first, then make sure you can route vm traffic, then start to look at VLAN, etc

@bobmc Yes, I figured it's going to be tricky with both needing to be up but for any maintenance, although the beauty of mobile phones these days and family shared data plans is hotspotting from phones to laptops and other devices in a pinch.

Thanks for the link to the guide and the advice on the process. Agreed, I'll waste an enormous amount of time troubleshooting if I try to build all at once and something isn't working.
I'm vry green with proxmox and VMs so just trying to get my head around the nuances of the bridging, gateways and subnets within proxmox to give the pfsense and VLANs the best chance of working but yes, I'll start with just the one network and build out from there.

Thx again

 

[imthiazaar]

I am pretty much trying to achieve the same..

@Gavcol - any breakthrough in configuration ?

I am trying to configure Sophos XG Home Firewall instead of PFsense.. on similar h/w.

 

[imthiazaar]

@Gavcol - Try this config - this solves the VM internet issue.
I am moving to second part.. of leveraging my WiFI router

 

[Gavcol]

I am pretty much trying to achieve the same..

@Gavcol - any breakthrough in configuration ?

I am trying to configure Sophos XG Home Firewall instead of PFsense.. on similar h/w.

Still working through it.
I have setup 6 linux bridges, one for each lan port and then added them in proxmox.

I does seem to work at first until I setup the vpn in pfsense with the same config I had before VMs but I'm having issues and trying to narrow down if it's to do with the linux bridges or maybe the AES-crypto (host cpu) or something else that's causing the issue.
Just working through a heap of process of elimination.

 

[Gavcol]

@Gavcol - Try this config - this solves the VM internet issue.
I am moving to second part.. of leveraging my WiFI router

Thanks @imthiazaar . Could you explain to me what you've done there ?

 

[imthiazaar]

I am pretty much done with my setup and now working on hardening the firewall.
My setup has only two bridges, but you can extend it as required.

vmbr1 - Upstream for Internet ( WAN )
vmbr2 - Downstream for LAN

I am routing my LAN traffic to vmbr1 - which forwards it to Internet. Followed the routed configuration steps stated in "https://pve.proxmox.com/wiki/Network_Configuration"

 

[dnphamus13]

On the small set-up (https://www.amazon.co.uk/dp/B08MW9WXS1) I have all network ports (HW pass-through) & the proxmox bridge going through pfsense. One physical port for WAN, and the remaining five physical ports & proxmox bridge is bridged by pfsense for LAN.

The dummy interface's IP address is the same as proxmox host to be able to access proxmox's webgui (& ssh) behind pfsense (LAN SIDE).

The first image shows the network interface config.
The second images ahows the hosts file config
The third images shows the pfsense VM config
The forth, fifth & sixth is from pfsense