Hey there,
I have this strange issue with the current Proxmox 8.1 SDN-module, where DHCP replies don't arrive at the destination VM.
I assume following setup:
Nodes are connected through Emulex SFP+ NICs, over a Mikrotik CRS315 Switch running SwOS (not RouterOS).
I assume that this setup works as expected, in that it pushes VLAN 90 onto all packages in vnet22A9, and every packet entering zone9539 receives VLAN Tag 834, so that the ethernet packet going out of eth0 is tagged as
What is not expected, is that DHCP Reply-packets aren't received across the switch.
That is:
- On HostA create a OpenWRT VM, with a NIC in vnet22A9 (192.0.2.1/24)
Setup a dhcp-server (in my case this is a dhcp relay, but that shouldn't matter)
- On HostB create a VyOS-VM (makes it easier to run renew commands), with a NIC in vnet22A9
- Configure VyOS with
- The OpenWRT VM sees the DHCP Discover broadcast on the vnet22A9-device
- In case of relay: Forwards the packet to relay, and receives a reply
- The OpenWRT VM creates a unicast DHCP Offer packet, and sends it out of the vnet22A9 device
- The VyOS VM never sees this DHCP Offer and continuously sends DHCP Discovers.
With that same setup, following works fine:
- Assign static ip to previously created VyOS VM
- From VyOS
- Ping to OpenWRT works
- traceroute -I to OpenWRT works
- traceroute to OpenWRT works
- From OpenWRT
- Ping to VyOS works
- traceroute -I to VyOS works
- traceroute to VyOS works
From my perspective this doesn't make any sense:
- From VyOS via CRS315 the OpenWRT VM sees the DHCP Discover
- From OpenWRT via CRS315 the VyOS VM doesn't see the DHCP Offer
- From OpenWRT via CRS315 Ping VyOS works
- From OpenWRT via CRS315 traceroute VyOS works
- From OpenWRT via CRS315 traceroute -I VyOS works
Additional things I observed:
- If VyOS and OpenWRT are colocated the DHCP Offer is received correctly
- If I change the zone VLAN protocol to 802.1ad the DHCP Offer is received
The CRS315 doesn't "know" the 802.1ad-protocol, thus treats the packet as untagged
I did try to replicate this scenario on one node, using the following setup:
Add OpenWRT to vmbr0 directly (allowing access to all VIDs)
Keep VyOS on the vnet22A9
In OpenWRT create bridge br0 over eth0, with VLAN Filtering enabled and adding VID 834 as a port
In OpenWRt create bridge br1 over br0.834, with VLAN Filtering enabled, and adding VID 90 as a port
Assign IP and DHCP Server to br1.90
This works fine, the DHCP client VyOS receives the DHCP Offer.
I did try things with
What's the reason, that 802.1q (ethtype 8100) stacked VLANs don't work across a physical switch? Does the switch really need to understand ethtype 8100 stacking, or is this some other option I missed entirely?
FWIW: I did use OpenVSwitch before, but the
I have this strange issue with the current Proxmox 8.1 SDN-module, where DHCP replies don't arrive at the destination VM.
I assume following setup:
Code:
# /etc/pve/sdn/zones.cfg
qinq: zone9539
bridge vmbr0
tag 834
ipam pve
vlan-protocol 802.1q
# /etc/pve/sdn/vnets.cfg
vnet: vnet22A9
zone zone9539
alias home-dhcp-test
tag 90
Code:
# /etc/network/interfaces
auto vmbr0
iface vmbr0 inet manual
bridge-ports eno1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
mtu 9000
# /etc/network/interfaces.d/sdn
auto ln_zone9539
iface ln_zone9539
link-type veth
veth-peer-name pr_zone9539
auto pr_zone9539
iface pr_zone9539
link-type veth
veth-peer-name ln_zone9539
auto vmbr0
iface vmbr0
bridge-vlan-protocol 802.1q
auto vnet22A9
iface vnet22A9
bridge_ports z_zone9539.90
bridge_stp off
bridge_fd 0
alias home-dhcp-test
auto z_zone9539
iface z_zone9539
bridge-stp off
bridge-ports vmbr0.834 ln_zone9539
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094Nodes are connected through Emulex SFP+ NICs, over a Mikrotik CRS315 Switch running SwOS (not RouterOS).
Code:
HostA
== 10G DAC ==
CRS315
== 10G DAC ==
HostBI assume that this setup works as expected, in that it pushes VLAN 90 onto all packages in vnet22A9, and every packet entering zone9539 receives VLAN Tag 834, so that the ethernet packet going out of eth0 is tagged as
0x8100 834 | 0x8100 90 | 0x0800.What is not expected, is that DHCP Reply-packets aren't received across the switch.
That is:
- On HostA create a OpenWRT VM, with a NIC in vnet22A9 (192.0.2.1/24)
Setup a dhcp-server (in my case this is a dhcp relay, but that shouldn't matter)
- On HostB create a VyOS-VM (makes it easier to run renew commands), with a NIC in vnet22A9
- Configure VyOS with
Code:
conf
set interface ethernet eth0 address dhcp- In case of relay: Forwards the packet to relay, and receives a reply
- The OpenWRT VM creates a unicast DHCP Offer packet, and sends it out of the vnet22A9 device
- The VyOS VM never sees this DHCP Offer and continuously sends DHCP Discovers.
With that same setup, following works fine:
- Assign static ip to previously created VyOS VM
Code:
conf
del interface ethernet eth0
set interface ethernet eth0 address 192.0.2.2/24- Ping to OpenWRT works
- traceroute -I to OpenWRT works
- traceroute to OpenWRT works
- From OpenWRT
- Ping to VyOS works
- traceroute -I to VyOS works
- traceroute to VyOS works
From my perspective this doesn't make any sense:
- From VyOS via CRS315 the OpenWRT VM sees the DHCP Discover
- From OpenWRT via CRS315 the VyOS VM doesn't see the DHCP Offer
- From OpenWRT via CRS315 Ping VyOS works
- From OpenWRT via CRS315 traceroute VyOS works
- From OpenWRT via CRS315 traceroute -I VyOS works
Additional things I observed:
- If VyOS and OpenWRT are colocated the DHCP Offer is received correctly
- If I change the zone VLAN protocol to 802.1ad the DHCP Offer is received
The CRS315 doesn't "know" the 802.1ad-protocol, thus treats the packet as untagged
I did try to replicate this scenario on one node, using the following setup:
Add OpenWRT to vmbr0 directly (allowing access to all VIDs)
Keep VyOS on the vnet22A9
In OpenWRT create bridge br0 over eth0, with VLAN Filtering enabled and adding VID 834 as a port
In OpenWRt create bridge br1 over br0.834, with VLAN Filtering enabled, and adding VID 90 as a port
Assign IP and DHCP Server to br1.90
This works fine, the DHCP client VyOS receives the DHCP Offer.
I did try things with
bridge-ageing 0 and bridge-learning off but that didn't make any difference.What's the reason, that 802.1q (ethtype 8100) stacked VLANs don't work across a physical switch? Does the switch really need to understand ethtype 8100 stacking, or is this some other option I missed entirely?
FWIW: I did use OpenVSwitch before, but the
dot1q-tunnel other-config:qinq-ethtype=802.1q didn't work either, so I switched everything back to Linux Bridges hoping that this would solve the issue.
Last edited:
