tagged vlan inguest

H

hdejongh

New Member
Jul 9, 2025
7
0
1
Hello,

Have been breaking my head for the past 6 hours on the following problem.
I have a VM (OPSense) in which i have an interface (connected to VMBR0), on that interface a created a vlan interface (vlan 10).
The vm has ip (10.8.10.254) and my core switch as well (10.8.10.200). Ping doesnt work (the firewall allows ping).

this is the output of my /etc/network/interfaces

Code: 
root@pve03:~# cat /etc/network/interfaces

# loopback
auto lo
iface lo inet loopback

# 25 Gbit

iface eno12399np0 inet manual
  mtu 9000
  post-up /sbin/ethtool -K eno12399np0 gro off

iface eno12409np1 inet manual
  mtu 9000
  post-up /sbin/ethtool -K eno12409np1 gro off


# bond
auto bond0
iface bond0 inet static
  bond-slaves eno12399np0 eno12409np1
  bond-mode 4
  bond-miimon 100
  bond-downdelay 200
  bond-updelay 200
  bond-lacp-active yes
  bond-lacp-rate fast
  bond-xmit-hash-policy layer3+4
  mtu 9000


# bond subinterfaces
auto bond0.10
  mtu 9000
auto bond0.15
  mtu 9000
auto bond0.20
  mtu 9000
auto bond0.21
  mtu 9000
auto bond0.50
  mtu 9000
auto bond0.51
  mtu 9000
auto bond0.60
  mtu 9000
auto bond0.61
  mtu 9000
auto bond0.756
  mtu 9000

# trunk

auto vmbr0
iface vmbr0 inet static
  bridge-ports bond0
  bridge-stp off
  bridge-fd 0
  mtu 9000
  bridge-vlan-aware yes
  bridge-vids 10-20

#Trunk

# management
auto vmbr10
iface vmbr10 inet static
  bridge-ports bond0.10
  bridge-stp off
  bridge-fd 0
  mtu 9000

#Management

# service
auto vmbr15
iface vmbr15 inet static
  bridge-ports bond0.15
  bridge-stp off
  bridge-fd 0
  address 10.8.15.13/24
  gateway 10.8.15.254
  mtu 9000
#Service

# proxmox migration
auto vmbr20
iface vmbr20 inet static
  bridge-ports bond0.20
  bridge-stp off
  bridge-fd 0
  address 10.8.20.13/24
  mtu 9000

#Proxmox - Migration

# proxmox ceph
auto vmbr21
iface vmbr21 inet static
  bridge-ports bond0.21
  bridge-stp off
  bridge-fd 0
  address 10.8.21.13/24
  mtu 9000

#Proxmox - Ceph


# lan
auto vmbr50
iface vmbr50 inet static
  bridge-ports bond0.50
  bridge-stp off
  bridge-fd 0
  mtu 9000
#LAN

# guest
auto vmbr51
iface vmbr51 inet static
  bridge-ports bond0.51
  bridge-stp off
  bridge-fd 0
  mtu 9000
#Guest


# security
auto vmbr60
iface vmbr60 inet static
  bridge-ports bond0.60
  bridge-stp off
  bridge-fd 0
  mtu 9000

#Security

# iot
auto vmbr61
iface vmbr61 inet static
  bridge-ports bond0.61
  bridge-stp off
  bridge-fd 0
  mtu 9000
#IoT


# wan
auto vmbr756
iface vmbr756 inet static
  bridge-ports bond0.756
  bridge-stp off
  bridge-fd 0
  mtu 9000

#WAN

source /etc/network/interfaces.d/*

output of: bridge -compressvlans vlan show

root@pve03:~# bridge -compressvlans vlan show
port vlan-id
tap110i0 1 PVID Egress Untagged
fwbr110i0 1 PVID Egress Untagged
fwln110i0 1 PVID Egress Untagged
tap110i1 1 PVID Egress Untagged
fwbr110i1 1 PVID Egress Untagged
fwln110i1 1 PVID Egress Untagged
bond0 1 PVID Egress Untagged
10-20
vmbr0 1 PVID Egress Untagged
bond0.10 1 PVID Egress Untagged
vmbr10 1 PVID Egress Untagged
bond0.15 1 PVID Egress Untagged
vmbr15 1 PVID Egress Untagged
bond0.20 1 PVID Egress Untagged
vmbr20 1 PVID Egress Untagged
bond0.21 1 PVID Egress Untagged
vmbr21 1 PVID Egress Untagged
bond0.50 1 PVID Egress Untagged
vmbr50 1 PVID Egress Untagged
bond0.51 1 PVID Egress Untagged
vmbr51 1 PVID Egress Untagged
bond0.60 1 PVID Egress Untagged
vmbr60 1 PVID Egress Untagged
bond0.61 1 PVID Egress Untagged
vmbr61 1 PVID Egress Untagged
bond0.756 1 PVID Egress Untagged
vmbr756 1 PVID Egress Untagged


tcpdump -i bond0 vlan 10 shows (when doing pinging from both sides at the same time)

tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on bond0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
14:35:44.467300 ARP, Request who-has 10.8.10.200 tell 10.8.10.254, length 28
14:35:44.842153 ARP, Request who-has 10.8.10.254 tell 10.8.10.200, length 46
14:35:45.288033 STP 802.1w, Rapid STP, Flags [Proposal, Learn, Forward, Agreement], bridge-id 800a.74:86:e2:c3:4a:a0.8a6b, length 42
14:35:45.522237 ARP, Request who-has 10.8.10.200 tell 10.8.10.254, length 28
14:35:45.896700 ARP, Request who-has 10.8.10.254 tell 10.8.10.200, length 46
14:35:46.577244 ARP, Request who-has 10.8.10.200 tell 10.8.10.254, length 28
14:35:46.996690 ARP, Request who-has 10.8.10.254 tell 10.8.10.200, length 46
14:35:47.307011 STP 802.1w, Rapid STP, Flags [Proposal, Learn, Forward, Agreement], bridge-id 800a.74:86:e2:c3:4a:a0.8a6b, length 42
14:35:47.628470 ARP, Request who-has 10.8.10.200 tell 10.8.10.254, length 28
14:35:48.096714 ARP, Request who-has 10.8.10.254 tell 10.8.10.200, length 46
14:35:48.668444 ARP, Request who-has 10.8.10.200 tell 10.8.10.254, length 28
14:35:49.197014 ARP, Request who-has 10.8.10.254 tell 10.8.10.200, length 46
14:35:49.324383 STP 802.1w, Rapid STP, Flags [Proposal, Learn, Forward, Agreement], bridge-id 800a.74:86:e2:c3:4a:a0.8a6b, length 42
14:35:49.708467 ARP, Request who-has 10.8.10.200 tell 10.8.10.254, length 28
14:35:50.737165 ARP, Request who-has 10.8.10.200 tell 10.8.10.254, length 28
14:35:51.341634 STP 802.1w, Rapid STP, Flags [Proposal, Learn, Forward, Agreement], bridge-id 800a.74:86:e2:c3:4a:a0.8a6b, length 42

I dont understand what i am doing wrong..
 
I see the vlan 10 on vmbr10, but what you've done about vmbr0 is making him able to trunk vlan 10 to 20. Are you aware about the mecanics between native and tagged Vlan ?
 
Specimen said:
I see the vlan 10 on vmbr10, but what you've done about vmbr0 is making him able to trunk vlan 10 to 20. Are you aware about the mecanics between native and tagged Vlan ?
Click to expand...
thanks,

VMBR10 is for servers who will send their traffic untagged, if they are in vmbr10 the traffic will be tagged (vlan tag 10) by the hypervisor.
VMBR0 is for servers who will send their traffic tagged (in this example the traffic will enter VMBR0 with vlan tag 10).

I need both.
 
Are you expecting your OPsense to receive tagged vlan 10 ? Because it wont. There is nothing which is likely to tag on vmbr0. it's all native. It needs to be tagged from above.
 
Last edited:
yes opsense needs to receive tagged traffic on vlan10. I dont expect VMBR0 to tag anything. I expect it to receive tagged vlan 10 traffic from bond0?
Am i making an thinking error?
 
switch from bvmbr0 to vmbr10 on your OPsense, and tell me if it pings. Have you ever tried the bond0 before using vlan ? Are you sure it works ?
 
it doesnt ofcourse;) cause opsense is sending out tagged traffic. But if i remove the tag from opsense and put it in vmbr10 it works fine.
Cause then the untagged traffic coming from opsense wil get tagged by the hypervisor with vmbr10.... do you understand general/tagged/access ports on switches:

VMBR0 should be trunk port
VMBR10 = access port.
 
Yes I understand, I had a doubt that the traffic was actually tagged when arriving from outside Proxmox, but I was missreading your logs. i'm processing on a possible issue.
 
hdejongh said:
it doesnt ofcourse;) cause opsense is sending out tagged traffic. But if i remove the tag from opsense and put it in vmbr10 it works fine.
Cause then the untagged traffic coming from opsense wil get tagged by the hypervisor with vmbr10.... do you understand general/tagged/access ports on switches:

VMBR0 should be trunk port
VMBR10 = access port.
Click to expand...
Okay, I made a little test here. I notices your tap110i0 (vm port) is not allowing vlan. You need to edit your vm network configuration on proxmox, and add the tag 10 on it. via :
Code: 
bridge vlan add dev tap110i0 vid 10
It does not work whitout it even if there is a network interface configured IN the VM. the tag will be blocked without that option.
 
Specimen said:
Okay, I made a little test here. I notices your tap110i0 (vm port) is not allowing vlan. You need to edit your vm network configuration on proxmox, and add the tag 10 on it. via :
Code: 
bridge vlan add dev tap110i0 vid 10
It does not work whitout it even if there is a network interface configured IN the VM. the tag will be blocked without that option.
Click to expand...
Doesnt work:
i used this commando:
bridge vlan add dev tap100i2 vid 10

so i thought about adding an IP to VMBR10 in the same subnet to rule out everything else..
Do you think that should work?

root@pve03:~# bridge -compressvlans vlan show
port vlan-id
bond0 1 PVID Egress Untagged
10
20
vmbr0 1 PVID Egress Untagged
bond0.10 1 PVID Egress Untagged
vmbr10 1 PVID Egress Untagged
bond0.15 1 PVID Egress Untagged
vmbr15 1 PVID Egress Untagged
bond0.20 1 PVID Egress Untagged
vmbr20 1 PVID Egress Untagged
bond0.21 1 PVID Egress Untagged
vmbr21 1 PVID Egress Untagged
bond0.50 1 PVID Egress Untagged
vmbr50 1 PVID Egress Untagged
bond0.51 1 PVID Egress Untagged
vmbr51 1 PVID Egress Untagged
bond0.60 1 PVID Egress Untagged
vmbr60 1 PVID Egress Untagged
bond0.61 1 PVID Egress Untagged
vmbr61 1 PVID Egress Untagged
bond0.756 1 PVID Egress Untagged
vmbr756 1 PVID Egress Untagged
tap100i0 1 PVID Egress Untagged
fwbr100i0 1 PVID Egress Untagged
fwpr100p0 1 PVID Egress Untagged
fwln100i0 1 PVID Egress Untagged
tap100i1 1 PVID Egress Untagged
fwbr100i1 1 PVID Egress Untagged
fwpr100p1 1 PVID Egress Untagged
fwln100i1 1 PVID Egress Untagged
tap100i2 1 PVID Egress Untagged
10
fwbr100i2 1 PVID Egress Untagged
fwpr100p2 1 PVID Egress Untagged
2-4094
fwln100i2 1 PVID Egress Untagged
root@pve03:~#
 
Last edited:
110 = test windows vm (i tried to add vlan 10 on the netwerkcard but read somewhere it doenst work with virtio in windows).
100 = OPsense.
 
Here is the test I made :

Pve1 : LXC ubuntu, untagged traffic -> vmbr1 (no vlan aware) -> Linux vlan 10 -> physical interface
Pve2 : LXC ubuntu, Vlan interface IN the lxc (eth0.10) -> bridge vlan aware -> physical interface.

First it didn't work. Then I made the command I gave you, and it worked.
 
im certain its not the vm, a TCPdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on tap100i2, link-type EN10MB (Ethernet), snapshot length 262144 bytes
20:43:38.050541 bc:24:11:46:e1:7c > ff:ff:ff:ff:ff:ff, ethertype 802.1Q-QinQ (0x88a8), length 46: vlan 10, p 0, ethertype ARP (0x0806), Request who-has 10.8.10.200 tell 10.8.10.253, length 28
20:43:39.099159 bc:24:11:46:e1:7c > ff:ff:ff:ff:ff:ff, ethertype 802.1Q-QinQ (0x88a8), length 46: vlan 10, p 0, ethertype ARP (0x0806), Request who-has 10.8.10.200 tell 10.8.10.253, length 28

(i im trying with a different iP now, .253).
the same is showing with a TCPDUMP on VMBR0

listening on vmbr0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
20:45:23.095168 ARP, Request who-has 10.8.10.200 tell 10.8.10.253, length 28
20:45:24.108843 ARP, Request who-has 10.8.10.200 tell 10.8.10.253, length 28
20:45:24.168094 STP 802.1w, Rapid STP, Flags [Proposal, Learn, Forward, Agreement], bridge-id 8001.74:86:e2:c3:4a:a0.8a6b, length 42
20:45:24.168165 STP 802.1w, Rapid STP, Flags [Proposal, Learn, Forward, Agreement], bridge-id 8001.74:86:e2:c3:4a:a0.8a6b, length 36
20:45:25.154946 ARP, Request who-has 10.8.10.200 tell 10.8.10.253, length 28

also on bond0 i see the same:

root@pve03:~# tcpdump -i bond0 -e -n vlan 10
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on bond0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
20:48:42.657937 bc:24:11:46:e1:7c > ff:ff:ff:ff:ff:ff, ethertype 802.1Q-QinQ (0x88a8), length 46: vlan 10, p 0, ethertype ARP (0x0806), Request who-has 10.8.10.200 tell 10.8.10.253, length 28
20:48:43.349680 74:86:e2:c3:4a:a0 > 01:00:0c:cc:cc:cd, ethertype 802.1Q (0x8100), length 68: vlan 10, p 7, 802.3LLC, dsap SNAP (0xaa) Individual, ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST (0x010b), length 42: STP 802.1w, Rapid STP, Flags [Proposal, Learn, Forward, Agreement], bridge-id 800a.74:86:e2:c3:4a:a0.8a6b, length 42
20:48:43.730440 bc:24:11:46:e1:7c > ff:ff:ff:ff:ff:ff, ethertype 802.1Q-QinQ (0x88a8), length 46: vlan 10, p 0, ethertype ARP (0x0806), Request who-has 10.8.10.200 tell 10.8.10.253, length 28
20:48:44.784177 bc:24:11:46:e1:7c > ff:ff:ff:ff:ff:ff, ethertype 802.1Q-QinQ (0x88a8), length 46: vlan 10, p 0, ethertype ARP (0x0806), Request who-has 10.8.10.200 tell 10.8.10.253, length 28
 
Last edited:
I figured it out ! Make a new vlan10 over your vmbr0. it will force vmbr0 to keep the tags.
here is an example I made :

auto vmbr90
iface vmbr90 inet manual
bridge-ports ens22
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 50-60

auto vmbr90.55
iface vmbr90.55 inet manual

I managed vlan inside a vm using vmbr90, and it didn't work until I created that vlan to force my tap103 to keep the traffic tagged.
 
Last edited:
You must log in or register to reply here.
Top Bottom