Dual NIC Proxmox v9 with OPNsense VM: how to configure for VLAN?

M-a-x-G

New Member
Apr 26, 2024
1
0
1
I understand this topic has been covered and I read a few of these threads; however there are mixed solutions; and what's worse, I can't get the following to work.

The tenor seems to be that OPNsense does not need to be configured for VLAN at all, when letting Proxmox taking care of it. With this premise, I post here and not in the OPNsense forum. VLANs make it possible to use several networks on the same physical link, which is what I'd like to achieve.

Proxmox has an OPNsense VM on it; fully configured and functioning without VLANs.

OPNsense sees this:
I have a PC with one NIC and an USB adapter for the WAN connection.
The PC runs Proxmox v9
- LAN (vtnet0) -> v4: 192.168.2.1/24
- WAN (vtnet1) -> will be a DHCP-issued public IP once connected to the Internet
- DMZ (vtnet2) -> v4: 192.168.3.1/24
- IoT (vtnet3) -> v4: 192.168.1.1/24

While Proxmox has this config
Code:
ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master vmbr0 state UP group default qlen 1000
    link/ether 00:23:24:a0:3c:3c brd ff:ff:ff:ff:ff:ff
    altname enp0s25
    altname enx002324a03c3c
3: enx00e04c896ad5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel master vmbr1 state DOWN group default qlen 1000
    link/ether 00:e0:4c:89:6a:d5 brd ff:ff:ff:ff:ff:ff
4: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:23:24:a0:3c:3c brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.2/24 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::223:24ff:fea0:3c3c/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
5: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:e0:4c:89:6a:d5 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::2e0:4cff:fe89:6ad5/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
6: vmbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 9e:3c:7c:d3:5f:65 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::40ed:d3ff:fe6f:c67e/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
7: vmbr3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether aa:6d:ba:67:10:21 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::d88c:b8ff:fef8:2458/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
8: vmbr4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:a8:c0:ca:5e:4a brd ff:ff:ff:ff:ff:ff
    inet6 fe80::f8a8:c0ff:feca:5e4a/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
9: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel master vmbr0 state UNKNOWN group default qlen 1000
    link/ether 4a:54:5d:ef:96:e1 brd ff:ff:ff:ff:ff:ff
10: tap100i1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel master vmbr1 state UNKNOWN group default qlen 1000
    link/ether ae:cf:8c:14:bb:89 brd ff:ff:ff:ff:ff:ff
11: tap100i2: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel master vmbr2 state UNKNOWN group default qlen 1000
    link/ether 9e:3c:7c:d3:5f:65 brd ff:ff:ff:ff:ff:ff
12: tap100i3: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel master vmbr3 state UNKNOWN group default qlen 1000
    link/ether aa:6d:ba:67:10:21 brd ff:ff:ff:ff:ff:ff
13: veth101i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether fe:bc:f6:04:b4:5b brd ff:ff:ff:ff:ff:ff link-netnsid 0
14: veth102i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr2 state UP group default qlen 1000
    link/ether fe:39:9e:50:0e:e3 brd ff:ff:ff:ff:ff:ff link-netnsid 1
15: veth103i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether fe:0d:37:e7:ab:ba brd ff:ff:ff:ff:ff:ff link-netnsid 2
16: veth104i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether fe:5c:5d:db:e4:08 brd ff:ff:ff:ff:ff:ff link-netnsid 3
17: veth105i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether fe:c4:2a:31:9b:b8 brd ff:ff:ff:ff:ff:ff link-netnsid 4

bridge vlan show
port              vlan-id 
eno1              1 PVID Egress Untagged
enx00e04c896ad5   1 PVID Egress Untagged
vmbr0             1 PVID Egress Untagged
vmbr1             1 PVID Egress Untagged
vmbr2             1 PVID Egress Untagged
vmbr3             1 PVID Egress Untagged
vmbr4             1 PVID Egress Untagged
tap100i0          1 PVID Egress Untagged
tap100i1          1 PVID Egress Untagged
tap100i2          1 PVID Egress Untagged
tap100i3          1 PVID Egress Untagged
veth101i0         1 PVID Egress Untagged
veth102i0         1 PVID Egress Untagged
veth103i0         1 PVID Egress Untagged
veth104i0         1 PVID Egress Untagged
veth105i0         1 PVID Egress Untagged

1759058649664.png

Proxmox is plugged into an unmanaged switch with a PC and notebook to ping and test.

What is recommended high level approach, and more so the order of things, to make this set-up using VLANs to isolate the networks?

I have prepared a 24-port managed switch tagged with VID 20 for the vmbr0 vtnet0 192.168.2.0/24 network. Port 23 is the trunk port, ports 13-18 have been configured for VLAN 20, where I can connect the PC and notebook to.
The switch is configured as follows:
Code:
Port Name    PVID    Mode    VLAN-allowed Range    VLAN-untagged Range        Operate
g0/13    20    Access    20    20        Edit   
g0/14    20    Access    20    20        Edit   
g0/15    20    Access    20    20        Edit   
g0/16    20    Access    20    20        Edit   
g0/17    20    Access    20    20        Edit   
g0/18    20    Access    20    20        Edit   
g0/23    1    Trunk    1-4094    none        Edit

Do I make all vmbrs VLAN-aware, or only vmbr0 with eno1 connected to it?
Do I create a bridge vmbr0.20 and also for the other networks, interfaces?

When I tried this before, I could only access Proxmox (192.168.2.2), not OPNsense (192.168.2.1).
Pleaee note: I only configured vmbr0.20 to see if I could get this to work

Code:
cat interfaces.vlan
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual
#NIC LAN

auto enx00e04c896ad5
iface enx00e04c896ad5 inet manual
#NIC (USB) WAN

auto vmbr0
iface vmbr0 inet static
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 20 30 40 50
#LAN

auto vmbr1
iface vmbr1 inet manual
    bridge-ports enx00e04c896ad5
    bridge-stp off
    bridge-fd 0
#WAN

auto vmbr2
iface vmbr2 inet manual
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
#DMZ

auto vmbr3
iface vmbr3 inet manual
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
#IoT

auto vmbr4
iface vmbr4 inet manual
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
#Guest

auto vmbr0.20
iface vmbr0.20 inet static
    address 192.168.2.2/24
    gateway 192.168.2.1
    vlan-raw-device vmbr0
#Proxmox VLAN 20

source /etc/network/interfaces.d/*

I am more than happy to provide any config to aid this case. Thank you.