rpi4 openwrt converting to proxmox opnsense

H

houseofrahl

New Member
Feb 27, 2024
16
0
1
on my pi4 i have openwrt configured with vlans, and trunked through a netgear gs308ep managed switch to a netgear wax214v2, i have it all working as is.

i would like to migrate to a i5 6400,8gb,256gb,i350-t4

id like to get my opnsense in proxmox setup to do the same, i am stuck when it comes to setting up my proxmox network connections, and i cant decide if im better off with passthrough nics or linux bridge, or linux vlans.

any advise or starting points?

my current setup has
isp modem -> rpi4 openwrt 2 usb nics
wan

vlan10 (lan) 192.168.10.1
vlan20 (iot) 192.168.20.1
vlan30(iso) 192.168.30.1

i dont have a normal untagged lan,

my switch is setup trunked on ports 1,2, and 3. (managed at 192.168.10.4)
port 1 is connected to the pi,
port 2 is connected to the AP,(managed at 192.168.10.8)
port 3 goes out to another managed switch
4-6 are vlan10,
7,8 vlan30,

vlan20 is only used on iot wifi

the new pc has a onboard port and a quad nic,
i want 1 lan and 1 wan port, i may expand later but for now i just want the one each
id like to manage proxmox on 192.168.10.16 (vlan 10) where i manage my switch and my ap also if that makes sense
id like to access opnsense at 192.168.10.1
id like all 3 vlans to trunk through lan

i am new to opnsense, but i take advise well, and im not opposed to google fu as long as theres some direction on what to find
 
Without passthrough it would look like this:

Code: 
auto lo
iface lo inet loopback

iface eno1 inet manual
#LAN
iface eno2 inet manual
#WAN
iface eno3 inet manual
iface eno4 inet manual
iface eno5 inet manual

auto vmbr0.10
iface vmbr0.10 inet static
        address  192.168.10.16/24
        gateway  192.168.10.1
# VLAN interface to give PVE an IP and gateway on VLAN10

auto vmbr0
iface vmbr0 inet manual
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#LAN bridge

auto vmbr1
iface vmbr1 inet manual
        bridge-ports eno2
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#WAN bridge
With OPNsense having virtio NICs attached to vmbr0 + vmbr1. Without setting a "VLAN Tag" for the virtio NIC in case you want to manage VLANs within OPNsense.

With passthrough you would need 3 NICs. 1 passthroughed for LAN, 1 passthroughed WAN and the third one not passthroughed for PVEs LAN access.
 
Last edited:
Dunuin said:
Without passthrough it would look like this:

Code: 
auto lo
iface lo inet loopback

iface eno1 inet manual
#LAN
iface eno2 inet manual
#WAN
iface eno3 inet manual
iface eno4 inet manual
iface eno5 inet manual

auto vmbr0.10
iface vmbr0.10 inet static
        address  192.168.10.16/24
        gateway  192.168.10.1
# VLAN interface to give PVE an IP and gateway on VLAN10

auto vmbr0
iface vmbr0 inet manual
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#LAN bridge

auto vmbr1
iface vmbr1 inet manual
        bridge-ports eno2
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#WAN bridge
With OPNsense having virtio NICs attached to vmbr0 + vmbr1. Without setting a "VLAN Tag" for the virtio NIC in case you want to manage VLANs within OPNsense.

With passthrough you would need 3 NICs. 1 passthroughed for LAN, 1 passthroughed WAN and the third one not passthroughed for PVEs LAN access.
Click to expand...
hey wow thank you, so i edit my etc/network/interfaces file and add this information in, create the opnsense and assign the nics as lan and wan?
 
houseofrahl said:
hey wow thank you, so i edit my etc/network/interfaces file and add this information in, create the opnsense and assign the nics as lan and wan?
Click to expand...
Yes, but edit it to match your NIC names and have a keyboard+display at hand in case we screwed something up and webUI+SSH is down. ;)
I would also recommend to backup your network config first: cp /etc/network/interfaces /etc/network/interfaces.backup
And don't forget to also edit /etc/resolv.conf to match your DNS server (192.168.10.1) and the /etc/hosts in case you change the PVE hosts IP (but could both also be done via webUI).
 
Last edited:
starting totally from a fresh no ethernet connceted proxmox install. so im at GROUND ZERO right now. i just did the initial setup and used 192.168.10.16 as the address, once i get in via the lan port ill nano the etc/network/interfaces file and see what my interfaces look like.
 
So I tried it, I ended up with the test computer getting an ip through opnsense, but I could no longer access proxmox. I went and looked at my file. And it doesn’t look like my settings took, so I’m gonna restore and try again.

When I edit the file, I press control and O, then enter to save it, do I need to do anything to apply it?
 
Tried again but this time I got locked out of everything. So I think I need to configure the 2 bridges first, setup opnsense with the vlans. Then finally change the management ip
 
houseofrahl said:
When I edit the file, I press control and O, then enter to save it, do I need to do anything to apply it?
Click to expand...
Either a reboot or restarting the network via systemctl restart networking.

houseofrahl said:
Tried again but this time I got locked out of everything. So I think I need to configure the 2 bridges first, setup opnsense with the vlans. Then finally change the management ip
Click to expand...
Would help if you could show the output of ip a as well as the text yu want to replace the /etc/network/interfaces with.
 
so this is basically stock, i can currently connect to either proxmox or opnsense depending on how i hook up the wires lol. (i deleted the wan bridge i had before and got myself back to near stock.

i have a spare managed switch with a trunk port and port vlans 10 20 and 30 and a spare pc to test it each step, i have not gotten the vlans to work yet, but i may be overthinking this, i could setup a lan and still have the vlans on top of it right? in openwrt i eliminated the default lan, but that may not have been necessary.

current network topography is (modem->rpi4 openwrt->trunk managed switch->portvlan20->proxmox machine) i would like to remove the rpi4 and put the proxmox machine there, and be able to access the proxmox interface, as well as have opnsense act as the firewall/router for the home network. the vlan config is secondary and mainly used to isloate my work laptop from my personal network (connected via ethernet) and give me the ability to put iot traffic on its own subnet,

if im overthinking it im open to alternate setups that accomplish the above goals. (im all about learning new things, opnsense looks to be way more powerful than openwrt and id like to get to know it. same with proxmox)

Code: 
auto lo
iface lo inet loopback

auto enp1s0f1
iface enp1s0f1 inet manual

auto enp1s0f0
iface enp1s0f0 inet manual

iface enp0s31f6 inet manual

iface enp1s0f2 inet manual

iface enp1s0f3 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.10.16/24
        gateway 192.168.10.1
        bridge-ports enp1s0f1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

source /etc/network/interfaces.d/*


Code: 
root@pve:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: enp1s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether a0:36:9f:90:ea:d0 brd ff:ff:ff:ff:ff:ff
3: enp1s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
    link/ether a0:36:9f:90:ea:d1 brd ff:ff:ff:ff:ff:ff
4: enp1s0f2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether a0:36:9f:90:ea:d2 brd ff:ff:ff:ff:ff:ff
5: enp1s0f3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether a0:36:9f:90:ea:d3 brd ff:ff:ff:ff:ff:ff
6: enp0s31f6: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 60:45:cb:a3:f1:e7 brd ff:ff:ff:ff:ff:ff
7: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether a0:36:9f:90:ea:d1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.16/24 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::a236:9fff:fe90:ead1/64 scope link
       valid_lft forever preferred_lft forever
 
houseofrahl said:
i have a spare managed switch with a trunk port and port vlans 10 20 and 30 and a spare pc to test it each step, i have not gotten the vlans to work yet, but i may be overthinking this, i could setup a lan and still have the vlans on top of it right?
Click to expand...
You usually want a single vlan-aware bridge for that trunk and then tag the VLANs for each VMs/LXCs virtual NIC and have all connected this same bridge.

houseofrahl said:
in openwrt i eliminated the default lan, but that may not have been necessary.
Click to expand...
No not necessary. You can mix tagged and untagged traffic. Having your LAN or management subnet untagged makes it often also easier when working with devices that don't offer to set up VLANs.

houseofrahl said:
opnsense looks to be way more powerful than openwrt
Click to expand...
It is.

What you set for your /etc/network/interfaces will tell PVE to get an IP via untagged traffic and PVE won't be part of a VLAN (so the default VLANID1 for non-VLAN-traffic) unless you tell your switch to also use untagged VLAN for your LAN. In that case the trunk would need to be set to use VLANID10 untagged + VLANID20 and 30 tagged.
If your switch got VLANID 10,20,30 tagged you would need to add that part and remove the IP/gateway from "vmbr0":
Code: 
auto vmbr0.10
iface vmbr0.10 inet static
        address  192.168.10.16/24
        gateway  192.168.10.1
# VLAN interface to give PVE an IP and gateway on VLAN10
 
Last edited:
would there be a more simplistic way to accomplish this?

if we forget the vlan configuration to begin with, and just get the initial setup going

main goal is isp modem->proxmox machine(opnsense router)->managed switch->devices and access point
ability to access webui of both proxmox and opnsense from the lan
 
Without any VLANs and two NICs (one connected as WAN to modem and one connected to a port of the switch with LAN as untagged or without any VLAN set) it would look like this:

Code: 
auto lo
iface lo inet loopback

iface enp0s31f6 inet manual
#unused NIC
iface enp1s0f0 inet manual
#WAN NIC
iface enp1s0f1 inet manual
#LAN NIC
iface enp1s0f2 inet manual
#unused NIC
iface enp1s0f3 inet manual
#unused NIC

auto vmbr0
iface vmbr0 inet static
        address 192.168.10.16/24
        gateway 192.168.10.1
        bridge-ports enp1s0f1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#LAN bridge

auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp1s0f0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#WAN bridge

source /etc/network/interfaces.d/*
With enp1s0f1 connected to switch and enp1s0f0 connected to modem.
 
  • Like
Reactions: houseofrahl
Dunuin said:
Without any VLANs and two NICs (one connected as WAN to modem and one connected to a port of the switch with LAN as untagged or without any VLAN set) it would look like this:

Code: 
auto lo
iface lo inet loopback

iface enp0s31f6 inet manual
#unused NIC
iface enp1s0f0 inet manual
#WAN NIC
iface enp1s0f1 inet manual
#LAN NIC
iface enp1s0f2 inet manual
#unused NIC
iface enp1s0f3 inet manual
#unused NIC

auto vmbr0
iface vmbr0 inet static
        address 192.168.10.16/24
        gateway 192.168.10.1
        bridge-ports enp1s0f1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#LAN bridge

auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp1s0f0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#WAN bridge

source /etc/network/interfaces.d/*
With enp1s0f1 connected to switch and enp1s0f0 connected to modem.
Click to expand...
so this is the proxmox side of it, internally for opnsense would i set the ip as the 192.168.10.1 and that would be the main dhcp for itself and proxmox at that point?
 
houseofrahl said:
so this is the proxmox side of it, internally for opnsense would i set the ip as the 192.168.10.1 and that would be the main dhcp for itself and proxmox at that point?
Click to expand...
Yes.
One virtio NIC connected to vmbr0 without setting a "VLAN Tag" to be used in OPNsense as LAN.
One virtio NIC connected to vmbr1 without setting a "VLAN Tag" to be used in OPNsense as WAN.
Setting up OPNsense to work as DNS/DHCP/NTP/gateway using 192.168.10.1 on LAN where your PVE and all other devices will connect to.
 
im off to try it out now, i think my issues previous here have been that my main network is using the same ip, so it was conflicting with itself. will update shortly, thank you very much for the time you've put in here!~ i will update in a bit on how its going
 
so im currently behind my existing network still, but im using that on the wan, and i now have dhcp working downstream! and i can access opnsense and proxmox from the lan, WOOT! thank you for that!, i would like to add vlans on top of it for vlan10,20,30 can i do this without taking down whats working right now?
 
houseofrahl said:
vlan10,20,30
Click to expand...
If your LAN is on the VLAN10 you shouldn't add this again as a VLAN (I guess you configured your switches port now to trunk VLAN10 untagged + VLAN 20 and 30 tagged). For VLAN 20 and 30 you can simply add "20" or "30" as "VLAN Tag" to the virtio NIC when creating a VM/LXC. OPNsense should also be able to work with all trunked VLANs in case you didn't set any VLAN Tag for its virtio NICs.
 
Dunuin said:
If your LAN is on the VLAN10 you shouldn't add this again as a VLAN (I guess you configured your switches port now to trunk VLAN10 untagged + VLAN 20 and 30 tagged). For VLAN 20 and 30 you can simply add "20" or "30" as "VLAN Tag" to the virtio NIC when creating a VM/LXC. OPNsense should also be able to work with all trunked VLANs in case you didn't set any VLAN Tag for its virtio NICs.
Click to expand...
well it will be replacing my current network going forward,
so i used a totally different subnet 192.168.40.1 in opnsense, and i used 192.168.40.4 in proxmox vmbr0 is my lan port, vmbr1 is my wan port (currently connected to my upstream network, but will replace upstream network)

so basically, i could turn off vlan aware on the wan port and run another set of vlans downstream in my current config right?
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom