Router on a Container Network Configuration

[Lemure]

Hi, I am new to Proxmox. I just installed it in my home server and played around with it a bit, created a few containers, changed the network configuration. I want to know if the network configuration that I am planning makes sense and it is safe, or if there is a better way to do what I want to do.

I am installing Proxmox in my home server. The only other physical devices are the ISP router and other consumer devices that will get an IP through DHCP. In the future, I want to run another Proxmox in a VPS as a reverse proxy for the services in my home server, so my home ISP IP is not exposed, but this is not my concern now. The Proxmox in my home server should run a container that will act as a router.

My home server has 3 network connections, 2 wired ethernet and one wireless. One of the wired ethernet will be for the host. The other wired ethernet will be for the router container and will be connected to the ISP router, to the internet. The wireless device will be for the router container and will act as an wireless AP.

I have created a diagram because writing it would be long and tedios to read (some doubts after the diagram):






1. Is this the best way to run a router as a container inside a Proxmox host. I would prefer for the router to be a container/VM and not the host itself.
2. As I understand, the host needs to have its own ethernet interface, so that if the router container has an issue, I can still access the host. Since I have physical access to the machine, I will leave this ethernet (eth0 in the diagram) disconnected, but if the router container stops responding, I could connect to the host by setting up a static IP 10.10.10.XX in my laptop and connecting it to eth0. Is this correct?
3. Why do some linux bridges need their own IP?

 

[ph0x]

This looks absolutely legit and is widely used.
If you give the bridge an address, then the host is on that net. If you don't, it just passes the traffic into the bridge.

 

[bobmc]

1. I would agree that running a firewall/router in a VM is the best way to go. You can do both routing and firewall on the host but installing something like pfSense as a VM is far easier to maintain in my view.
2. Again, I think it makes logical sense for the VM to have a dedicated port for WAN but it's not technically necessary. Doesn't make much difference to VM/Host accessibility. This would mean that vmbr0 would be both LAN and Management port
3. The bridge can have it's own IP so you can access the host via that physical connection

Question would be if the virtual firewall/router is just for virtual/wireless traffic or for physical clients as well?

 

[Lemure]

Thanks for both answers.

2. Again, I think it makes logical sense for the VM to have a dedicated port for WAN but it's not technically necessary. Doesn't make much difference to VM/Host accessibility. This would mean that vmbr0 would be both LAN and Management port

How would one do this, having vmbr0 be both LAN and Management port? I would still like to access the host from inside the firewall.

Question would be if the virtual firewall/router is just for virtual/wireless traffic or for physical clients as well?

At the moment only virtual and wireless. In the future I might need to get another wired ethernet adapter to connect wired devices.

Also in the future I would like to have a VPS acting as a reverse proxy for the services running in this home server without exposing my home IP. I was thinking on having the VPS run Proxmox too and connecting it to the 10.10.10.XX network through a VPN connection between the VPS and the Router container (or is there a better way?) and having the VPS Proxmox be for example 10.10.10.5 . Is there a better way of doing this? Is it somehow dangerous having the VPS having access to my local network (I guess I could set up VLAN's once I have more services)?

 

[bobmc]

In your proposed setup, your physical LAN devices would be firewalled from the VM's so you would need establish firewall rules to allow access. Danger with this is that anything malicious that has got past your ISP router will then have the same access as well, making pfsense almost redundant.

You may want to consider making the 10.10.10.x network your 'LAN network' and route all traffic through the pfsense VM. That way you will get the maximum protection benefit from Intrusion Detection and Prevention packages like Snort or Suritica

The downside of this is that any maintenance to proxmox needs to be planned, otherwise you will not be popular when the internet goes down because you have to reboot the server

 

[Lemure]

In your proposed setup, your physical LAN devices would be firewalled from the VM's so you would need establish firewall rules to allow access. Danger with this is that anything malicious that has got past your ISP router will then have the same access as well, making pfsense almost redundant.

You may want to consider making the 10.10.10.x network your 'LAN network' and route all traffic through the pfsense VM. That way you will get the maximum protection benefit from Intrusion Detection and Prevention packages like Snort or Suritica

Well, what you describe was the intention. I can see I got wrong the IP of the eth1 interface, as there is where I have to set up the masquerade.

In fact, I realized that the eth0 interface does not need to be only for the host, it can be just for the bridge inside the internal network. So let me rebuild the whole diagram to see if I got it right, because this is still quite new to me.

The downside of this is that any maintenance to proxmox needs to be planned, otherwise you will not be popular when the internet goes down because you have to reboot the server

Yes, I am aware of that. I plan on running Home Assistant in this machine too, so when I have to update the whole house will stop working...

 

[Lemure]

Ok, diagram update.

@bobmc I think we both got it wrong. That bridge should have no IP, because the masquerading and firewalling should happen inside the Router Container/VM. I saw this by following a link you recommended here in the forum in another place that I read when I was trying to understand all of this, it is this: https://docs.netgate.com/pfsense/en...-proxmox-ve.html#virtualizing-with-proxmox-ve . In that tutorial, the bridges coming in and out of the Pfsense VM have no IP, because the masquerading and firewalling happens in Pfsense, not in the bridge.

Also, I have realized that the wifi adapter should be passthrough to the Router Container/VM so it can set up the AP, not a bridge like I had in the first version. Although I would love for someone with experience to confirm this.

This is the updated diagram (one big doubt/question after):



I think the Vbrm0 bridge does not need an IP address. But in the default installation of Proxmox the Vbrm0 bridge has the same IP as the host, which confuses me. It is working fine like that in my server, but I do not understand how it is working. So I would appreciate a clarification on how the Host and Vbrm0 can have the same IP. And if in the case of this updated diagram, if it is ok for Vbrm0 to have no IP.

 

[bobmc]

In the documented process the host has three ethernet adapters but you should note that vmbr0 does have an ip address.

When you install proxmox, the ip address your specify during the installation will automatically be assigned to vmbr0 by default. In your example, vmbr0 should have the 10.10.10.2 address. You should also use the IP address you plan to assign to pfsense as your default gateway - i.e 10.10.10.1 and this should also probably be your default dns - although you could continue to use your existing dns server if required.

Install pfsense with two network interfaces, one assigned to vmbr0 will be your LAN connection, the second you should assign to vmbr1 and this should be connected to your ISP router. Configure 10.10.10.1 as the LAN ip. The WAN ip can be static or assigned by dhcp, whichever you prefer.
You will need to allow private IP addresses on the WAN interface as this will be disabled by default in pfSense - see Interfaces/WAN/Reserved Networks. pfSense has a dhcp/dns server built in so you will probably need to set these up now.

At this point your proxmox host should be able to access the internet via pfsense.

Not sure about your plan to use the wifi adapter in the host as your access point, I doubt it will have much range or performance as it will be optimised for reception and won't have very much grunt to provide wifi services to mobile clients.

Wireless clients will still be able to access the internet via your isp router and will be isolated from your VM's which may be sufficient. Otherwise I'd sugggest purchasing a dedicated wireless AP. Used Unifi/Ubiquiti units are readily available on ebay if you don't want to spend too much, (you will also need a power adapter or a PoE capable network switch)

Good luck

 

[Lemure]

In the documented process the host has three ethernet adapters but you should note that vmbr0 does have an ip address.

When you install proxmox, the ip address your specify during the installation will automatically be assigned to vmbr0 by default. In your example, vmbr0 should have the 10.10.10.2 address. You should also use the IP address you plan to assign to pfsense as your default gateway - i.e 10.10.10.1 and this should also probably be your default dns - although you could continue to use your existing dns server if required.

Install pfsense with two network interfaces, one assigned to vmbr0 will be your LAN connection, the second you should assign to vmbr1 and this should be connected to your ISP router. Configure 10.10.10.1 as the LAN ip. The WAN ip can be static or assigned by dhcp, whichever you prefer.
You will need to allow private IP addresses on the WAN interface as this will be disabled by default in pfSense - see Interfaces/WAN/Reserved Networks. pfSense has a dhcp/dns server built in so you will probably need to set these up now.

At this point your proxmox host should be able to access the internet via pfsense.

Agreed.

Althouhg I would like to understand why the Vmbr0 bridge needs to have the IP of the Proxmox host. If anyone can answer I would be thankful.

Not sure about your plan to use the wifi adapter in the host as your access point, I doubt it will have much range or performance as it will be optimised for reception and won't have very much grunt to provide wifi services to mobile clients.

Wireless clients will still be able to access the internet via your isp router and will be isolated from your VM's which may be sufficient. Otherwise I'd sugggest purchasing a dedicated wireless AP. Used Unifi/Ubiquiti units are readily available on ebay if you don't want to spend too much, (you will also need a power adapter or a PoE capable network switch)

Good luck

This is a home server and my house is not that big. Plus the adapter has good signal. I have used this adapter as AP before in another house and had no problems. In my experience you need to make sure to get an adapter with good drivers for linux, and it will work fine. The one I have the drivers are included in the kernel. I know people everywhere recomends using a dedicated AP, but I have had no problem using dongles as AP.

Anyway, seems like tomorrow I will give a try to this configuration. Hopefully we got it right. Thanks for your time.

 

[ph0x]

It's just the standard installation routine that the host has the ip address in vmbr0 as his primary address. Everything is configurable but this way means the least hassle.

 

[Lemure]

It's just the standard installation routine that the host has the ip address in vmbr0 as his primary address. Everything is configurable but this way means the least hassle.

If I understand this correctly, the bridge has the proxmox host IP to indicate that the proxmox host is using that bridge. For the VM and containers you can manually asign the bridges they are using, so that is not necessary. It is a convention. Thanks for the explanation.

 

[ph0x]

It's not really an indicator but more the host's address in said bridge. You can change addresses (to a certain extent) and bridges how you like it, but with the installer assigning the primary address to vmbr0 it's the most convenient to leave it like that.

 

[Lemure]

This worked. Thanks everybody. I am updating the diagram with the small fix of the IP address, just in case anyone checks this in the future. Also, note that at first the host will not have Internet access, so you need a VM like pfsense ready or in my case previously create a container template with masquerading and dnsmasq so you can set up the router container without internet.

 

[lavaquark]

Install pfsense with two network interfaces, one assigned to vmbr0 will be your LAN connection, the second you should assign to vmbr1 and this should be connected to your ISP router

Sorry for re-opening this: I'm trying to create a similar setup. Do I have to set up pfsense to route all traffic from vmbr0 to the router and consequently vmbr1, or can this be set up with no additional tools, like ip routes etc.? And how and where do I have to configure this route?