Can't get SDN VLAN Isolation to work

A

AristonPost

Member
Feb 15, 2024
21
5
8
Zurich
Topic says it all. But first let me introduce the topology:

PVE Host connected via LACP (bond0) to a core switch pair (QSW-M7308R-4X) MLAG trunk port. Trunk port has PVID 1 and IDs 1600 and 1702 configured. This configuration is known to work as there's an Access Switch connected exactly the same way that serves other VLANs (100,101) very well.

The Gateway (VyOS) is connected via another Switch that's also connected to the core switch and configured for VLANs 100,101 and 1702.

Code: 
$ sh int
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface    IP Address                    MAC                VRF        MTU  S/L    Description
-----------  ----------------------------  -----------------  -------  -----  -----  ----------------------
...
eth4         172.16.0.1/24                 58:47:ca:7d:54:df  default   1500  u/u    LAN1
eth4.100     172.16.100.1/24               58:47:ca:7d:54:df  default   1500  u/u    Clients local.lan
eth4.101     172.16.101.1/24               58:47:ca:7d:54:df  default   1500  u/u    Guests local.lan
eth4.1702    172.17.2.1/24                 58:47:ca:7d:54:df  default   1500  u/u    Teddycloud Application
...

PVE has SDN VLAN configured

Code: 
# cat interfaces
...

auto lo
iface lo inet loopback

auto nic0
iface nic0 inet manual

auto nic1
iface nic1 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves nic0 nic1
        bond-miimon 100
        bond-mode 802.3ad
        bond-xmit-hash-policy layer2+3
        mtu 1500
        bond-lacp-rate 1

auto vmbr0
iface vmbr0 inet static
        address 172.16.0.254/24
        gateway 172.16.0.1
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0

source /etc/network/interfaces.d/*

Code: 
# cat interfaces.d/sdn

#version:3

auto ln_net1600
iface ln_net1600
        link-type veth
        veth-peer-name pr_net1600
        mtu 1500

auto ln_net1702
iface ln_net1702
        link-type veth
        veth-peer-name pr_net1702
        mtu 1500

auto net1600
iface net1600
        bridge_ports ln_net1600
        bridge_stp off
        bridge_fd 0
        mtu 1500

auto net1702
iface net1702
        bridge_ports ln_net1702
        bridge_stp off
        bridge_fd 0
        mtu 1500

auto pr_net1600
iface pr_net1600
        link-type veth
        veth-peer-name ln_net1600
        mtu 1500

auto pr_net1702
iface pr_net1702
        link-type veth
        veth-peer-name ln_net1702
        mtu 1500

auto vmbr0v1600
iface vmbr0v1600
        bridge_ports  bond0.1600 pr_net1600
        bridge_stp off
        bridge_fd 0
        mtu 1500

auto vmbr0v1702
iface vmbr0v1702
        bridge_ports  bond0.1702 pr_net1702
        bridge_stp off
        bridge_fd 0
        mtu 1500

There's one container running configured to use net1702

Code: 
# pct list
VMID       Status     Lock         Name
100        running                 teddycloud.local.lan

Code: 
# pct config 100
arch: amd64
cores: 2
features: nesting=1
hostname: teddycloud.local.lan
memory: 512
nameserver: 172.16.0.1
net0: name=eth0,bridge=net1702,gw=172.17.2.1,hwaddr=BC:24:11:CE:62:28,ip=172.17.2.15/24,type=veth
ostype: centos
rootfs: local-lvm:vm-100-disk-0,size=8G
searchdomain: local.lan
swap: 512
unprivileged: 1

PVE Host networking:
Code: 
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: nic0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
    link/ether 50:7c:6f:4e:a4:cc brd ff:ff:ff:ff:ff:ff
    altname enp94s0f0np0
    altname enx507c6f4ea4cc
3: nic1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
    link/ether 50:7c:6f:4e:a4:cc brd ff:ff:ff:ff:ff:ff permaddr 50:7c:6f:4e:a4:cd
    altname enp94s0f1np1
    altname enx507c6f4ea4cd
4: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether 50:7c:6f:4e:a4:cc brd ff:ff:ff:ff:ff:ff
5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 50:7c:6f:4e:a4:cc brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.254/24 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::527c:6fff:fe4e:a4cc/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
8: net1600: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether da:d2:07:a0:d4:28 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::d8d2:7ff:fea0:d428/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
20: net1702: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 42:3e:d9:4f:92:2d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::403e:d9ff:fe4f:922d/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
31: veth100i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master net1702 state UP group default qlen 1000
    link/ether fe:de:82:b7:17:0b brd ff:ff:ff:ff:ff:ff link-netnsid 0
40: ln_net1600@pr_net1600: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master net1600 state UP group default qlen 1000
    link/ether da:d2:07:a0:d4:28 brd ff:ff:ff:ff:ff:ff
41: pr_net1600@ln_net1600: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0v1600 state UP group default qlen 1000
    link/ether da:25:9b:ca:14:86 brd ff:ff:ff:ff:ff:ff
48: ln_net1702@pr_net1702: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master net1702 state UP group default qlen 1000
    link/ether 42:3e:d9:4f:92:2d brd ff:ff:ff:ff:ff:ff
49: pr_net1702@ln_net1702: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0v1702 state UP group default qlen 1000
    link/ether 22:f2:34:ef:73:8d brd ff:ff:ff:ff:ff:ff
50: bond0.1600@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0v1600 state UP group default qlen 1000
    link/ether 50:7c:6f:4e:a4:cc brd ff:ff:ff:ff:ff:ff
51: vmbr0v1600: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 50:7c:6f:4e:a4:cc brd ff:ff:ff:ff:ff:ff
    inet6 fe80::527c:6fff:fe4e:a4cc/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
58: bond0.1702@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0v1702 state UP group default qlen 1000
    link/ether 50:7c:6f:4e:a4:cc brd ff:ff:ff:ff:ff:ff
59: vmbr0v1702: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 50:7c:6f:4e:a4:cc brd ff:ff:ff:ff:ff:ff
    inet6 fe80::527c:6fff:fe4e:a4cc/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever

And here's what happens:
  • Container can not reach it's Gateway
  • Gateway can not reach Container
  • Gateway can be reached from a Testsystem (Windows 11) in VLAN 1702
  • Gateway can also be reached from VLAN 100,101 by various Clients

So for me this Looks like VLAN Isolation in General is not working on this PVE host but I don't get it where to look for.By the way... It's PVE 9.2.2 installed from 9.2-1 ISO on a Dell T440 system. network card is an Intel 810 with dual SFP28.
 
Last edited:
Generally it's recommended to enable VLAN-awareness on bridges that back VLAN zones, which avoids creating a lot of additional virtual network interfaces. You'll need to reapply the SDN configuration afterwards.

Did you try using tcpdump to check whether traffic is leaving the node correctly and returning correctly? That might give an indication on what's amiss in your network configuration.
 
You must log in or register to reply here.
Top Bottom
Back