Shouldn't ProxMox warn if VLAN is used without "VLAN Aware"?

[MrPete]

I just caused myself some serious grief, breaking our network for hours. I've solved the problem... but I believe ProxMox could easily help users to avoid this issue.

A bit of background

• I have a four port (I350) Intel network card with SR-IOV capability. It's been wonderful.
• One of the ports is used as a multi-VLAN trunk by pfSense. Until now, ProxMox didn't need to know anything about that. It just works.

The problem

• I wanted to give another VM access to one of the VLANs
• I added a (virtio) network device, on the proper VLAN
• Didn't seem to do anything, so I removed it again
• My entire network was busted. Rebooting pfSense didn't help. Packets would go in one direction but not the other, and other strange effects.
• The only solution: reboot the ProxMox host

The ACTUAL problem
Turns out, to use a VLAN on a VM network device you MUST:

• Check "VLAN Aware" when configuring the VM bridge on the ProxMox host
• reboot the host ("apply configuration" is not enough)

Two Key Questions

• Is there any valid purpose for using a VLAN id on a VM network device, without setting "VLAN Aware"?
• Wouldn't it make sense to warn users when a VLAN id is used, without VLAN Aware checked?
• (In fact, perhaps VLAN Aware could be auto-checked... and a warning given that the ProxMox host may need a reboot!)

 

[spirit]

without vlan aware, it's working too. It's just that the tagging in done on the physical interface instead the bridge.

something like, for example vlan10

with a default conf like

eth0--> vmbr0 (non vlan-aware)

setting a vm on vmbr0, with tag=10, proxmox will create in background:

eth0--->eth0.10---->vmbr0v10 ----->vm

(the tag is set when going out though eth0.10)

But this block vlan10 to vmbr0 if you had a vm with a trunk , like pfsense.
(as the vlan10 packet is now fowarded to vmbr0v10)

 

[MrPete]

I can say, from painful experience doing a pile of WireShark/tcpdump scans at various physical points points on the network:

If I have:
* a trunked vmbrX connection with no VLAN aware, used just as a trunk (works just fine, any number of VLAN's set up by pfSense there)
* and I build and destroy another instance on vmbrX using a VLAN for another VM
* THEN, it's almost random whether packets will go out or be seen for any given VLAN, until I reboot the host with the extra instance gone.
(SOME VLANs worked just fine. Some were completely broken)

I checked: inside the pfSense VM, in a monitor of the trunk, in a port defined to capture each VLAN after trunk breakout.

So "block vlan10 to vmbr0" is actually random. And not useful.

 

[spirit]

to be honest, I never had tried trunk with non-vlanaware bridge. (I thinked that vlan tag was dropped).

 

[Xtra]

I just caused myself some serious grief, breaking our network for hours. I've solved the problem... but I believe ProxMox could easily help users to avoid this issue.

A bit of background

• I have a four port (I350) Intel network card with SR-IOV capability. It's been wonderful.
• One of the ports is used as a multi-VLAN trunk by pfSense. Until now, ProxMox didn't need to know anything about that. It just works.

The problem

• I wanted to give another VM access to one of the VLANs
• I added a (virtio) network device, on the proper VLAN
• Didn't seem to do anything, so I removed it again
• My entire network was busted. Rebooting pfSense didn't help. Packets would go in one direction but not the other, and other strange effects.
• The only solution: reboot the ProxMox host

The ACTUAL problem
Turns out, to use a VLAN on a VM network device you MUST:

• Check "VLAN Aware" when configuring the VM bridge on the ProxMox host
• reboot the host ("apply configuration" is not enough)

Two Key Questions

• Is there any valid purpose for using a VLAN id on a VM network device, without setting "VLAN Aware"?
• Wouldn't it make sense to warn users when a VLAN id is used, without VLAN Aware checked?
• (In fact, perhaps VLAN Aware could be auto-checked... and a warning given that the ProxMox host may need a reboot!)

I've spent 3 days trying to figure out why opnsense running on proxmox wouldn't connect to my new ISP that needed VLAN 911 tagging to get a WAN IP.
You sir are a legend.