QinQ setup

[troycarpenter]

Greetings all.

I have two VM guests that communicate with each other using various VLANs, which requires a bit of trunking configuration with the intervening network equipment.

Using QinQ, what I would like to do is configure the system so that ALL outgoing guest packets get tagged with an outer VLAN ID which will be used in network. Using a router that can generate such QinQ packets, I have seen that simply adding the VLAN tag in the guest's network configuration is sufficient for receiving packets...that is, the outer VLAN tag is removed and the inner one remains. However, for outgoing packets it appears to either tag the untagged before sending, or simply trunks the existing tagged packets.

Is there a flag or option that I can set that will have the VM host always add the extra tag to packets originating from the guest VM? The end result should be that untagged packets from the guest will have the single tag added, but already tagged packets will also get the tag added for QinQ.

And, of course, the other guests on the host should not get the tag...I need this on a per guest basis.

Thanks

 

[Richard]

Regarding QinQ have a look here: https://forum.proxmox.com/threads/vlan-aware-option.25640

 

[troycarpenter]

Thanks. I'm working on this in the background with other tasks. That's a page I've been looking at, but not been able to get the time to move ahead. I also moved my interface from openvswitch back to linux bridge. It looks like openvswitch is still working on patches to get QinQ to work.

 

[troycarpenter]

Now a few months later, and after looking at the link above, I'm ready to try to get this working. However, I need to understand some things from that thread. This is going to get long, so please bear with me.

First, it looks like I need to turn off the vlan-aware option on interfaces for this to work.

Second, it would appear that I need to create the outer VLAN tag as a network interface directly in the /etc/network/interfaces file. I have an 8 node cluster that is using HA. This seems to imply that whenever I need to create a new guest that will utilize QinQ, then I need to add a new VLAN entry to every node's interfaces file, which is less than ideal and a hassle. It's already hard enough to synchronize the interfaces file to ensure a VM can startup on the new node if it gets moved.

Thirdly, I don't see how this would actually work because the way it is being described, it will confuse the guests that are using this scheme. Let me give a network diagram just in case I'm not understanding this properly.

As stated earlier, I have a 8 node cluster, where each node is a blade in a chassis, which for networking means that I cannot use any external Ethernet ports that may be on the blades so that VM migrations will work.

The guest systems are all essentially residential router type systems, where the "LAN" port now has no separate physical interface. I will be using VLANs to trunk various ports from a network switch to the virtualized "LAN" port on the guest, where each guest instance utilizes a unique VLAN (usually the same as the VMID in Proxmox to make it easier to recognize). This configuration was easy to setup.

Now the wrinkle. I want to connect a wireless access point that tags its SSIDs with VLANs to identify a client subnet. These VLANs only exist between the AP and the LAN port of the guest router software. These SSID VLANs are NOT unique. So for a VM with an ID of 2000, I want to use QinQ tagging to take the non-unique VLAN (say 800) generated by the AP, have the switch tag it with an outer tag of 2000, and pass it to the cluster chassis where the appropriate node will see the 2000, remove the tag, then send the packet to the guest VM with the 800 tag intact.

I have everything working up to the Proxmox node. I can verify with tcpdump on the node that the dual tagged QinQ packets are arriving on the appropriate interface.

My question is how do I get the node to take off the outer tag (2000) and pass the single tagged (800) packet to the guest. AND, if I need to have the outer tag defined as an interface on all my nodes, how to I prevent the non-uniquely tagged packet from going to all VM instances instead of just the one I want?

(And yes, I can make this work if I use unique VLANS for everything, but that requires touching lots of networking equipment and user education that I would rather avoid)

 

[spirit]

Hi,
as Far I remember QinQ is working :

-with linux classic bridge only (no vlan aware, no openswitch).

you need simply to define a vmbr with

eth.X-----vmbr0

then put vm interface on vmbr0, with tagged vlan in vm nic option. (Y)

Like type you'll have Y vlan in X vlan.