Proper or best practice way to set-up VLANs on single NIC?

M

M-a-x-G

New Member
Apr 26, 2024
7
0
1
I have now spend endless days what should be a simple problem; and gave up, now asking: what is the best way to set up VLANs.

The way I understand it (and I am new to VLANs), is to set a trunk port on the switch (I have set VIDs 20 30 40 50) and access port for VID 20 (=LAN); Proxmox is 192.168.2.2 and has vmbr0 tagged with VIDs 20 30 40 50. This should allow my PC to access the Proxmox GUI. Is this correct?

The minute I connect both Proxmox .2 and my PC .100 to the tagged ports, I can neither ping .2 nor get to the Proxmox GUI. I have a notebook on this VLAN too .164, and can ping it and vice versa.

What on earth am I missing? (I am not using the Proxmox firewall, and th eOPNsense VM is shut down; just to test this basic network.)

My switch is set up like this:
1759275366431.png

My Proxmox config ie:
Code: 
# [2025-10-01 09:40] root@pve1 ~ #
brctl show
bridge name    bridge id        STP enabled    interfaces
vmbr0        8000.002324a03c3c    no        eno1
                            veth103i0
                            veth104i0
                            veth105i0
vmbr1        8000.00e04c896ad5    no        enx00e04c896ad5
vmbr2        8000.fe13c64dffd6    no        veth102i0
vmbr3        8000.000000000000    no      
vmbr4        8000.000000000000    no      

# [2025-10-01 09:40] root@pve1 ~ #
cat /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual
#NIC LAN

auto enx00e04c896ad5
iface enx00e04c896ad5 inet manual
#NIC (USB) WAN

auto vmbr0
iface vmbr0 inet static
    address 192.168.2.2/24
    gateway 192.168.2.1
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 20 30 40 50
#LAN

auto vmbr1
iface vmbr1 inet manual
    bridge-ports enx00e04c896ad5
    bridge-stp off
    bridge-fd 0
#WAN

auto vmbr2
iface vmbr2 inet manual
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 30
#DMZ

auto vmbr3
iface vmbr3 inet manual
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 40
#IoT

auto vmbr4
iface vmbr4 inet manual
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 50
#Guest

source /etc/network/interfaces.d/*

or the GUI equivalent...
1759305638900.png
 
Last edited:
If PVE shall listen explicitly on VLAN XX (for management purposes or similar) you should create a dedicated VLAN vmbr for this, vmbr0.20 (where 0 is the bridge ID followed by the VLAN tag) and assign the desired ip.
 
VLANs can be tricky to debug. If your switch supports it, give it an IP in the VLAN, then you can check if the connection to the switch works.
For guests, by now I recommend that you use the SDN VLAN zone. It is one easy place to have every VLAN. In a cluster it makes life a lot easier, but also on a single node IMHO.

If you need to place the hosts mgmt IP in a VLAN and don't want to pass it as the untagged one on the switch (nomenclature might differ between switches), then you don't need a bridge (vmbrX) but can just add a new Linux VLAN device.
Code: 
auto mgmt
iface mgmt inet static
    address a.b.c.d/e
    gateway a.b.c.f
    vlan-raw-device eno1
    vlan-tag {VLAN ID}

If you add it via the GUI, the result would be similar. The vmbr0 would then not necessarily be VLAN aware (SDN can handle both) and won't have any address or gateway config.

I also don't know what you try to achieve with all the other vmbrX interfaces that don't have a bridge port. A bridge is a virtual switch that guests can be connected to. To give them a connection to the physical network, you need to set one physical interface as bridge port. (A bond if you need redundancy).

If you have multiple physical NICs that should be used for different traffic (not redundancy, e.g. bond) then you would need to define multiple SDN VLAN zones, as you define one bridge as the base interface per zone.
 
  • Like
Reactions: M-a-x-G
readyspace said:
Just a quick question - Is the bridge in Proxmox VE set to VLAN aware?
Click to expand...
yes, I updated my O.P.


In general: my understanding is (which could be wrong) that I do not need to mess with VLANs in OPNsense if I let Proxmox deal with the 'tag/untag' work. This way OPNsense see these interfaces as untagged traffic.
 
Last edited:
OK, so I updated by interfaces file as follows:
Code: 
auto lo
iface lo inet loopback

iface eno1 inet manual
# NIC LAN (trunk to g0/23)

auto enx00e04c896ad5
iface enx00e04c896ad5 inet manual
#WAN via USB adaptor

auto vmbr0
iface vmbr0 inet manual
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 20
# LAN (VLAN 20)

auto vmbr2
iface vmbr2 inet manual
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 30
# DMZ (VLAN 30)

auto vmbr3
iface vmbr3 inet manual
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 40
# IoT (VLAN 40)

auto vmbr4
iface vmbr4 inet manual
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 50
# Guest (VLAN 50, inactive for now)

auto vmbr1
iface vmbr1 inet manual
    bridge-ports enx00e04c896ad5
    bridge-stp off
    bridge-fd 0
# WAN (untagged, IP from ISP via DHCP later)

auto vmbr0.20
iface vmbr0.20 inet static
    address 192.168.2.2/24
    gateway 192.168.2.1
# Proxmox management on VLAN 20

source /etc/network/interfaces.d/*

It includes the suggestion from
cwt said:
reate a dedicated VLAN vmbr for this, vmbr0.20
Click to expand...

And the Gui version looks like this:
1759310221126.png

At least I can get to Proxmox now 2.2, but can't get to OPNsense on 2.1 after I started it.

What is a new observation for me: eno1 now appears as port on each bridge.
 
Having multiple vmbr interfaces with the same physical bridge port doesn't sound like a good idea. I would definitely recommend that you set up a SDN VLAN zone with vmbr0 as the base bridge for it and go from there for all the VLANs that should be accessible by the guests.
 
  • Like
Reactions: M-a-x-G
aaron said:
Having multiple vmbr interfaces with the same physical bridge port doesn't sound like a good idea. I would definitely recommend that you set up a SDN VLAN zone with vmbr0 as the base bridge for it and go from there for all the VLANs that should be accessible by the guests.
Click to expand...
Yes, I figured... now using this config:

1759320463042.png

still working on it...
 

Attachments

  • 1759316485654.png
    1759316485654.png
    41.5 KB · Views: 6
looks better :)

the subnets are useful to define as the info there will pre-populate firewall aliases.

you then need to apply it on the main SDN panel
 
aaron said:
looks better :)

the subnets are useful to define as the info there will pre-populate firewall aliases.

you then need to apply it on the main SDN panel
Click to expand...
Yes, thanks...

What puzzles me is why VID1 is shown in the interfaces file:

Code: 
bridge vlan show | head -20
port              vlan-id
enx00e04c896ad5   1 PVID Egress Untagged
vmbr0             1 PVID Egress Untagged
                  20
                  30
                  40
                  50
vmbr1             1 PVID Egress Untagged
veth103i0         1 PVID Egress Untagged
                  2
                  3
                  4
                  5
                  6
                  7
                  8
                  9
                  10
                  11
                  12

I though Proxmox should tag frames (20, 30, 40, 50) via SDN VNets, but the PVID 1 egress setting is overriding, sending untagged VLAN 1 to OPNsense. OPNsense expects untagged traffic per the current design. Is the design wrong? Does OPNsense need the VIDs? I had this running on an unmanaged switch and all worked (Proxmox and OPNsense, ping, connects and all). Now I can't get to OPNsense.
I am new to VLANs and Proxmox, so a ll a bit daunting ATM.
 
Last edited:
VLAN tag 1 is usually the default untagged VLAN. As in, those packets won't get a VLAN tag added when they leave the physical interface.

But whenever you assign one of the SDN VNETs to a guets virtual NIC, any packet leaving the host should get the set VLAN tag assigned. One can check that with Wireshark/TCPdump.

Not sure what you setup is in relation to OPNsense. But if you want to have the VLANs also terminate there, so you can do routing between the VLANs, you have some options. The simplest one would be to add new virtual NICs to the OPNSense VM, one per VLAN. They should show up as new interface within OPNSense. Use the MAC address to make sure you map the virtual NICs correctly to the interfaces OPNSense sees.
 
  • Like
Reactions: M-a-x-G
Thanks for hanging in there :)
... I do appreciate it.

I may completely misunderstand the whole setup; my initial idea was to have Proxmox deal with the single NIC and multiple subnets, by tagging these for the switch to digest and dish out. This was the vmbr approach.
This SDN approach seems different, here we use the same vmbr, 0 in this case, and VLAN ID tag it. This is what I have got now:

1759382886819.png
 
You must log in or register to reply here.
Top Bottom