VLAN-Aware Bridge with physical access-ports

[lonestar]

Hey,

I have a simple home-setup with an ISP-provided router and one Proxmox node. I do not have a managed switch, but I still want to use VLANs to separate virtual ressources on my node.

I have created a VLAN-aware bridge and added workloads to it, including a pfSense appliance that holds all VLANs from that bridge in a trunk (by not selecting a VLAN Tag in the VirtIO network adapter).

I now want to use physical interfaces to work as access-ports to different VLANs on this bridge. One use-case for this is to connect a laptop directly to my PVE node and have it participate in my admin-VLAN for desaster recovery purposes.

Using enp87s0 (which is slaved to vlan-aware bridge vmbr1) as an access port to VLAN 182 can be achieved using:

bridge vlan add dev enp87s0 vid 182 pvid untagged

... which works perfectly, but it doesn't persist.

Does anyone know how to implement this in /etc/networks/interfaces? Or any better idea how to achieve this?

 

[shanreich]

In /etc/network/interfaces uou can define an IP on the interface vmbr1.182 like so:

auto vmbr1.182
iface vmbr1.182 inet static
        address 10.0.0.1/24

 

[lonestar]

Hey Stephan,

thanks for the response! I already have this set up, in order to give my PVE node an IP in this VLAN.
However, this doesn't automatically make enp87s0 an "access port". For clarity, I want to attach a "client" device to this NIC. Basically making the Ethernet ports of my PVE node work like a normal managed switch.

Here is my current /etc/network/interfaces (vmbr0 is my normal, untagged LAN from ISP router. vmbr1 is where I'm trying to achieve this).

auto lo
iface lo inet loopback

iface enp88s0 inet manual

iface enp87s0 inet manual

iface enp2s0f0np0 inet manual

iface enp2s0f1np1 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.REDACTED/24
        gateway 192.168.REDACTED
        bridge-ports enp88s0
        bridge-stp off
        bridge-fd 0

iface wlp89s0 inet manual

auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp87s0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

auto vmbr1.182
iface vmbr1.182 inet static
        address 192.168.182.REDACTED/24

and after reboot, bridge -compressvlans vlan show results in:

# bridge -compressvlans vlan show
port              vlan-id
enp87s0           1 PVID Egress Untagged
                  2-4094
enp88s0           1 PVID Egress Untagged
vmbr0             1 PVID Egress Untagged
vmbr1             1 PVID Egress Untagged
                  182
tap100i0          1 PVID Egress Untagged
fwbr100i0         1 PVID Egress Untagged
fwpr100p0         1 PVID Egress Untagged
fwln100i0         1 PVID Egress Untagged

whereas after my "bridge vlan add dev enp87s0 vid 182 pvid untagged" command, it shows:

port              vlan-id
enp87s0           1 Egress Untagged
                  2-181
                  182 PVID Egress Untagged
                  183-4094
enp88s0           1 PVID Egress Untagged
vmbr0             1 PVID Egress Untagged
vmbr1             1 PVID Egress Untagged
                  182
tap100i0          1 PVID Egress Untagged
fwbr100i0         1 PVID Egress Untagged
fwpr100p0         1 PVID Egress Untagged
fwln100i0         1 PVID Egress Untagged

notice the "182 PVID Egress untagged" on the enp87s0 interface. That's what I want to achieve. The bridge now forwards traffic from VLAN 182 untagged out to enp87s0. Vice versa, untagged traffic from enp87s0 is tagged with VLAN 182 and properly forwarded only to my VMs that are in that VLAN. Any device that I attach to enp87s0 now functions just like a VM whose virtual network card is set to VLAN Tag 182.

 

[shanreich]

Ah sorry, I though you just wanted an IP on that VLAN. In this case you can bridge vmbr1.182:

auto vlan128
iface vlan128
        bridge-ports vmbr1.128
        bridge-stp off
        bridge-fd 0

You should also be able to simply do this via SDN, create a VLAN Zone [1] with vmbr1 as bridge and then create a VNet with Tag 128 and pick a name. Make sure vmbr1 is vlan-aware (which it currently is anyway). Then you should be able to select the VNet you created

[1] https://pve.proxmox.com/pve-docs/chapter-pvesdn.html#pvesdn_zone_plugin_vlan
[2] https://pve.proxmox.com/pve-docs/chapter-pvesdn.html#pvesdn_config_vnet

 

[spirit]

try to add

"bridge-pvid 128" on vmbr1

 

[lonestar]

Ah sorry, I though you just wanted an IP on that VLAN. In this case you can bridge vmbr1.182:

auto vlan128
iface vlan128
        bridge-ports vmbr1.128
        bridge-stp off
        bridge-fd 0

You should also be able to simply do this via SDN, create a VLAN Zone [1] with vmbr1 as bridge and then create a VNet with Tag 128 and pick a name. Make sure vmbr1 is vlan-aware (which it currently is anyway). Then you should be able to select the VNet you created

[1] https://pve.proxmox.com/pve-docs/chapter-pvesdn.html#pvesdn_zone_plugin_vlan
[2] https://pve.proxmox.com/pve-docs/chapter-pvesdn.html#pvesdn_config_vnet

I'll give that a try. At which point in your example though is enp87s0 specifically assigned as access port / untagged? Should I slave it to the iface vlan128 you mention?

To reiterate, I would like my ProxMox node to function like a regular Cisco / HP / whatever switch. maybe that's a tall order, and I'm definitely considering just getting a hardware switch. But for science - I basically want a bridge device in Linux (representing the "backplane" of the switch) and then decide, for each of my physical NICs, what VLANs they will carry and what the PVID of that port is. I also want to attach VMs to that switch, and decide whether they will participate in a certain VLAN or just receive a trunk port.

Excuse my bad drawing, but this is what I'm looking for:

 

[spirit]

you can have a look at cumulus linux documentation (it's an hardware switch using linux bridge with ifupdown2, exactly like proxmox)

https://docs.nvidia.com/networking-...hernet-Bridging-VLANs/VLAN-aware-Bridge-Mode/

auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp87s0 enp88s0 enp89s0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

auto enp87s0
iface enp87s0
    bridge-access 182

auto enp88s0
iface enp88s0
    bridge-access 181

auto enp89s0
iface enp89s0
  bridge-vids 181 182

 

[lonestar]

Thank you, the bridge-access stanza on the enp87s0 did the trick! I didn't know that one could configure this on the slaved interface!

I think we can consider this solved. For cosmetics - where would I put this config ideally? I was thinking an extra file /etc/network/interfaces.d/custom-accessports with contents

iface enp87s0 inet manual
        bridge-access 182

... or could this conflict with anything?

 

[spirit]

Thank you, the bridge-access stanza on the enp87s0 did the trick! I didn't know that one could configure this on the slaved interface!

I think we can consider this solved. For cosmetics - where would I put this config ideally? I was thinking an extra file /etc/network/interfaces.d/custom-accessports with contents

iface enp87s0 inet manual
        bridge-access 182

... or could this conflict with anything?

yes, no problem, I'll be merged with config from /etc/network/interfaces