how to set vxlan tunnel as trunk? vlan mapping to vxlan vni?

kitecn · May 18, 2022

i read redhat https://developers.redhat.com/artic...mmands-and-features#spanning_tree_protocol,it say
Since 4.11, the kernel has provided a native way to support VxLAN bridging. The topology for this looks like Figure 7. The vxlan0 endpoint in this figure was added with lightweight tunnel (LWT) support to handle multiple VNIs.
then,many vlan can Carried by one vxlan tunnel , vxlan like trunk, use vlan id mapping to vxlan vni
host1 on pve7 has two nics,ens18(192.168.100.206,link to host2),ens19 (trunk vlan 300,10)
Linuxversion : localhost.localdomain 5.10.0-60.18.0.50.oe2203.x86_64 #1 SMP Wed Mar 30 03:12:24 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
config as

systemctl stop firewalld
#Recipe 2: create all your netdevs
# create bridge device:
ip link add bridge type bridge vlan_filtering 1
ip link add vxlan0 type vxlan external local 192.168.100.206 dstport 4789
#ip link add vxlan0 type vxlan id 300 dev ens18 dstport 4789
# enslave local and remote ports
ip link set dev vxlan0 master bridge
ip link set dev ens19 master bridge
#Recipe 2: Enable vlan filtering and vlan_tunnel mode
#configure vlan filtering on bridge
#ip link set dev bridge type bridge vlan_filtering 1
# enable tunnel mode on the vxlan tunnel bridge ports
#bridge vlan show
bridge link set dev vxlan0 vlan_tunnel on
#Recipe 2: configure vlans
#configure vlans
bridge vlan add vid 10 dev vxlan0
bridge vlan add vid 10 dev ens19
bridge vlan add vid 300 dev vxlan0
bridge vlan add vid 300 dev ens19
# set tunnel mappings on the ports per vlan
# map vlan 10 to tunnel id 10 (in this case vni 10)
bridge vlan show
bridge vlan add dev vxlan0 vid 10 tunnel_info id 10
bridge vlan add dev vxlan0 vid 300 tunnel_info id 300
#Recipe 2: configure default fdb entries
# add your default remote dst forwarding entry
bridge fdb add 00:00:00:00:00:00 dev vxlan0 vni 300 dst 192.168.100.160 self permanent
ip link set bridge up
ip link set vxlan0 up mtu 1450
bridge fdb show
bridge -j -p vlan tunnelshow

host2 also have two nics, ens18(192.168.100.106),ens19(trunk vlan 300,10)
systemctl stop firewalld
#Recipe 2: create all your netdevs
# create bridge device:
ip link add bridge type bridge vlan_filtering 1
# create vxlan netdev:
ip link add vxlan0 type vxlan external local 192.168.100.160 dstport 4789
# enslave local and remote ports
ip link set dev vxlan0 master bridge
ip link set dev ens19 master bridge
#Recipe 2: Enable vlan filtering and vlan_tunnel mode
#configure vlan filtering on bridge
# ip link set dev bridge type bridge vlan_filtering 1
# enable tunnel mode on the vxlan tunnel bridge ports
#bridge vlan show
bridge link set dev vxlan0 vlan_tunnel on
#Recipe 2: configure vlans
#configure vlans
#bridge vlan add vid 10 dev vxlan0
#bridge vlan add vid 10 dev ens19
bridge vlan add vid 300 dev vxlan0
bridge vlan add vid 300 dev ens19
# set tunnel mappings on the ports per vlan
# map vlan 10 to tunnel id 10 (in this case vni 10)
# bridge vlan add dev vxlan0 vid 10 tunnel_info id 10
bridge vlan add dev vxlan0 vid 300 tunnel_info id 300
#Recipe 2: configure default fdb entries
# add your default remote dst forwarding entry
# bridge fdb add 00:00:00:00:00:00 dev vxlan0 vni 10 dst 192.168.100.206 self permanent
bridge fdb add 00:00:00:00:00:00 dev vxlan0 vni 300 dst 192.168.100.206 self permanent
ip link set bridge up
ip link set vxlan0 up mtu 1450
bridge vlan show
bridge fdb show
bridge -j -p vlan tunnelshow

but it's fail, vlan300 at ens19 can't through vxlan0 ....
i Catch the Packet，
at host 1
tcpdump -i ens19 -v -w ens19_vxlan.cap

Frame 1: 64 bytes on wire (512 bits), 64 bytes captured (512 bits)
Encapsulation type: Ethernet (1)
Arrival Time: May 18, 2022 19:25:26.288185000
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1652873126.288185000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 64 bytes (512 bits)
Capture Length: 64 bytes (512 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:vlan:ethertype:arp]
[Coloring Rule Name: ARP]
[Coloring Rule String: arp]
Ethernet II, Src: HuaweiTe_a2:cb:72 (0c:c6:cc:a2:cb:72), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
Destination: Broadcast (ff:ff:ff:ff:ff:ff)
Address: Broadcast (ff:ff:ff:ff:ff:ff)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
Source: HuaweiTe_a2:cb:72 (0c:c6:cc:a2:cb:72)
Address: HuaweiTe_a2:cb:72 (0c:c6:cc:a2:cb:72)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN, PRI: 6, DEI: 0, ID: 300
110. .... .... .... = Priority: Internetwork Control (6)
...0 .... .... .... = DEI: Ineligible
.... 0001 0010 1100 = ID: 300
Type: ARP (0x0806)
Padding: 0000000000000000000000000000
Trailer: 00000000
Address Resolution Protocol (request)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: request (1)
Sender MAC address: HuaweiTe_a2:cb:72 (0c:c6:cc:a2:cb:72)
Sender IP address: 192.168.30.10
Target MAC address: 00:00:00_00:00:00 (00:00:00:00:00:00)
Target IP address: 192.168.30.11

it has 802.1q,the vlan is 300

tcpdump -i vxlan0 -v -w vxlan0_vxlan.cap

Frame 1: 60 bytes on wire (480 bits), 60 bytes captured (480 bits)
Ethernet II, Src: HuaweiTe_a2:cb:72 (0c:c6:cc:a2:cb:72), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
Destination: Broadcast (ff:ff:ff:ff:ff:ff)
Source: HuaweiTe_a2:cb:72 (0c:c6:cc:a2:cb:72)
Type: ARP (0x0806)
Padding: 000000000000000000000000000000000000
Address Resolution Protocol (request)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: request (1)
Sender MAC address: HuaweiTe_a2:cb:72 (0c:c6:cc:a2:cb:72)
Sender IP address: 192.168.30.10
Target MAC address: 00:00:00_00:00:00 (00:00:00:00:00:00)
Target IP address: 192.168.30.11

802.1q vlanid has dropped, is normal packet , is not vxlan packet.

but the vlan mapping to vxlanvni is not effect.

what's the matter?

spirit · May 18, 2022

you can use the proxmox sdn feature to configure vxlan

https://pve.proxmox.com/pve-docs/chapter-pvesdn.html

Currently you can create 1bridge (vnet in sdn) for each vni. (1vni/1vxlan tunnel)

or you can also create 1 vlan-aware vxlan vnet , and passthrough vlanid (define on vm nic) inside the vxlan tunnel.

I don't have implemented yet lightweight tunnel, where each vlan tag is mapped to 1vni in 1 vxlan tunnel.
but you can implemented in in /etc/network/interfaces
https://docs.nvidia.com/networking-...mplifies the,with multiple VLAN-aware bridges.

kitecn · May 19, 2022

spirit said:
you can use the proxmox sdn feature to configure vxlan

https://pve.proxmox.com/pve-docs/chapter-pvesdn.html

Currently you can create 1bridge (vnet in sdn) for each vni. (1vni/1vxlan tunnel)

or you can also create 1 vlan-aware vxlan vnet , and passthrough vlanid (define on vm nic) inside the vxlan tunnel.

I don't have implemented yet lightweight tunnel, where each vlan tag is mapped to 1vni in 1 vxlan tunnel.
but you can implemented in in /etc/network/interfaces
https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-44/Network-Virtualization/VXLAN-Devices/#:~:text=Single VXLAN device simplifies the,with multiple VLAN-aware bridges.

thanks, i think it only for Cumulus Linux,cann't apply to other linux like centos, the shell script i posted Yesterday aslo from cumulus 。

spirit · May 19, 2022

kitecn said:
thanks, i think it only for Cumulus Linux,cann't apply to other linux like centos, the shell script i posted Yesterday aslo from cumulus 。

oh, do you talk about mounting vxlan inside a vm ?

I thinked you was asking about vxlan setup on proxmox ve hypervisor.(mounting the vxlan tunnel on hypervisor, and transport vlan tags into your vms)

avalentin · Oct 31, 2023

Hi!
I managed to get vxlan with vlan-vni mapping to work:

Code:

auto vx2
iface vx2 inet manual
        pre-up /sbin/ip -6 link add ${IFACE} type vxlan external novnifilter dstport 4789 learning df unset dev vmbr0 group ff02::201
        pre-up /sbin/ip link set dev ${IFACE} up
        post-down /sbin/ip link del dev ${IFACE}

auto vmbr2
iface vmbr2
        bridge-stp off
        bridge-fd 0
        bridge-mcsnoop 0
        bridge-vids 1-4094
        bridge-vlan-aware yes
        bridge-ports vx2
        post-up /sbin/bridge link set dev vx2 vlan_tunnel on
        post-up /sbin/bridge vlan add dev vx2 vid 1-4094 tunnel_info id 1-4094
        post-up /bin/sh -c 'for vid in $(seq 1 4094); do echo fdb add 00:00:00:00:00:00 dev vx2 dst ff02::201 via vmbr0 port 4789 self permanent src_vni $vid; done | /sbin/bridge -force -batch -'
        mtu 1500

This generates a VXLAN interface and a bridge. All VLANs are mapped to a VNI with the same value.
It uses IPv6 multicast to do the learning of MAC adresses. Therefore you need an interface, here vmbr0, with IPv6 link local addresses and a switch infrastructure with usable multicast support. In this example, I selected ff02::201 as the multicast group. It will correspond to all proxmox servers. This should be very efficient. After learning phase, the communication is switched to unicast.
All VLANs/VNI are separated and have their own MAC address space. If you want multiple VXLANs/bridges, just select a different port instead of 4789.
A big advantage of this solution is that no extra services are needed, everything is already there.

Of course your vmbr0 interface should have a bigger MTU, i.e. 9000, a minimum of 1570 is recommended to get a vxlan interface with MTU 1500.

Search

Search

how to set vxlan tunnel as trunk? vlan mapping to vxlan vni?

kitecn

New Member

spirit

Distinguished Member

kitecn

New Member

spirit

Distinguished Member

avalentin

New Member