VLAN management

C

casalicomputers

Renowned Member
Mar 14, 2015
89
3
73
Hi there,
My test host has 2 GbE nics bonded together through 802.3ad, and the switch is configured in a way that the bonded ports can access untagged default vlan and some tagged vlans (10,20,30) for different purposes. Coming from ESXi, I'm quite confused on how to deal with VLANs with proxmox, so here are my questions:

1) Is it possible to create different bridges using the same single bond?
For example I would like to create a bridge (eg. vmbr30) which is tagged to a specific VLAN by default (like configured tagged vswitches on esxi).
Is this possible?

2) By default, from guest VMs, I'm able to create virtual interfaces and access any vlan the host has access to.
Is there a way to limit which VLANs can be accessed from a particular VM or Bridge?

I've been able to achieve this by using openvswitch and specifying which ports a specific tap* interface could access
Code: 
root@pve01:~# ovs-vsctl set port tap100i0 vlan_mode=trunk
root@pve01:~# ovs-vsctl set port tap100i0 trunk=0,10
In this way I've been able to limit the guest to access default untagged vlan, and tagged vlan 10 but not others. The only problem is that this configuration is not persistent across host reboots.

3) Is there any difference about using openvswitch over traditional linux bridging? Which is the best choice in your opinion?



Of course, I'm aware that I could create several virtual nics and assign the desired vlan tag to it, but for some application I need to manage tagging from inside the VM using one interface only.


Thanks,
Michele
 
Last edited:
Since OpenvSwitch was introduced I would use it all the time instead of the native linux model.

The easiest way to get it working is to build your proxmox hosts and get them on a network using the simple standard linux networking model. Then using the proxmox GUI, delete all the Linux network interfaces and create yourself an OVS Bond, Bridge and OVSIntPort for host comms.

It means you just have a single bridge on which to connect everything and specific the VLAN tag to use when creating any vNIC's on VM's or IntPorts for the host. This means you can trunk extra VLAN's to the host in the future and put VM's onto them without any extra configuration / reboots.

If you used the native linux model you would have to create a vlan interface + bridge for each vlan, and connect each vNIC to those bridges individually. If you want to get extra VLAN's through to your VM's then you will need to create a new vlan int + bridge then give the hosts a reboot.
 
Thanks for your reply.
Could you please provide an example of /etc/network/interfaces about tagging with OVS IntPorts?
And.. what if I would like to have a bridge with tag=0 (untagged) ? is this possible?
 
casalicomputers said:
Thanks for your reply.
Could you please provide an example of /etc/network/interfaces about tagging with OVS IntPorts?
And.. what if I would like to have a bridge with tag=0 (untagged) ? is this possible?
Click to expand...

Sure, here is the interface config file from one of my hosts:

Code: 
# network interface settings
allow-vmbr0 mgmtvlan10
iface mgmtvlan10 inet static
    address  10.89.7.1
    netmask  255.255.255.0
    gateway  10.89.7.254
    ovs_type OVSIntPort
    ovs_bridge vmbr0
    ovs_options tag=10


auto lo
iface lo inet loopback


iface eth0 inet manual


iface eth1 inet manual


auto eth2
iface eth2 inet static
    address  192.168.94.1
    netmask  255.255.255.0
    mtu 9000


auto eth3
iface eth3 inet static
    address  192.168.95.1
    netmask  255.255.255.0
    mtu 9000


allow-vmbr0 bond0
iface bond0 inet manual
    ovs_bonds eth0 eth1
    ovs_type OVSBond
    ovs_bridge vmbr0
    ovs_options lacp=active bond_mode=balance-tcp


auto vmbr0
iface vmbr0 inet manual
    ovs_type OVSBridge
ovs_ports bond0 mgmtvlan10

As you can see, my bridge 'vmbr0' is an untagged bridge to my LACP bond. I havn't tried adding a VM without a tag but I would guess that you can just create a vNIC on this bridge without specifying a tag which would allow you to apply the tagging inside the VM. (Note that you have to specifiy any OVSIntPorts in the bridge as well).

My OVSIntPort is tagged to my management VLAN which is the hosts network connection. Remember, an OVSIntPort is for the host only. This is not for VM communication.

One thing to note however... When I was setting my 16 hosts up with Open vSwitch, even if I wrote out the above config manually in the interfaces file and reboot, the host didn't seem to recognise them as OVS interfaces. But when I set all the OVS stuff up in the GUI, it worked fine.

My eth2 and eth3 interfaces are for my iSCSI network and are not using Open vSwitch.
 
Last edited:
Hi,
thanks for your config file, but I see you're just using a single bridge (vmbr0) for vm connectivity.
Probably I didn't have fully understood yet how OVS works (still reading...), but I was looking to see if the following configuration can be made:

bond0 --> eth0, eth1
vmbr0(bond0) vlan_mode=access, tag=0 --> VMs on this bridge are NOT able to access any VLAN (eg. using vconfig)
Code: 
vm# vconfig add eth0.10
vm# ifconfig eth0.10 10.10.10.8 netmask 255.255.255.0 up
vm# ping 10.10.10.1 <-- this should not work!
vmbr1(bond0) vlan_mode=trunk,tag=0,10 --> VMs on this bridge are by default on untagged vlan 1 (tag=0) but CAN access tagged vlan10

Hope you got the point.
 
Last edited:
casalicomputers said:
Hi,
thanks for your config file, but I see you're just using a single bridge (vmbr0) for vm connectivity.
Probably I didn't have fully understood yet how OVS works (still reading...), but I was looking to see if the following configuration can be made:

bond0 --> eth0, eth1
vmbr0(bond0) vlan_mode=access, tag=0 --> VMs on this bridge are NOT able to access any VLAN (eg. using vconfig)
vmbr1(bond0) vlan_mode=trunk,tag=0,10 --> VMs on this bridge are by default on untagged vlan 1 (tag=0) but CAN access tagged vlan10

Hope you got the point.
Click to expand...

Hi,

You are thinking about this wrong. With Open vSwitch, you create a single bridge onto your bond. You then bridge your VM NIC's onto this bridge and specifiy the VLAN tag when you set up the vNIC. You could create multiple vNIC's and put each of them on a different VLAN... Or you might be able to not specify any tags so you can control tagging inside a VM.

However it sounds like you are trying to use the native untagged VLAN as well which I have not done before in Open vSwitch because I can't think of a reason why you would ever want to. I did read up though that you can place a bridge into 'native-untagged' mode though which might work... But you will need to read up yourself.... Or perhaps a better idea would be to consider why you are using the native untagged vlan?
 
I was pretty sure to be wrong.

Well, I have a VM running some UTM software, which can't see more than 4 physical nics, but allows the use of unlimited vlans, so this is one reason.
The other reason is: since bridges are trunks by default, this works out of the box, but I don't like the idea that anyone with root access on any other vm except the UTM, could access any VLAN simply by setting a tag.
I also noticed that the proxmox gui don't like vlan tag set to 0 directly on vm nics....

So the question could be:
how do I forbid a vm to access tagged vlans? and how do I allow some other to access them, instead?
Obviously I'm always talking about tagging vlan from guest VMs.


What about OVSPatchPort to interconnect bridges and using vlan tags on them?

Thanks
 
casalicomputers said:
I was pretty sure to be wrong.

Well, I have a VM running some UTM software, which can't see more than 4 physical nics, but allows the use of unlimited vlans, so this is one reason.
The other reason is: since bridges are trunks by default, this works out of the box, but I don't like the idea that anyone with root access on any other vm except the UTM, could access any VLAN simply by setting a tag.
I also noticed that the proxmox gui don't like vlan tag set to 0 directly on vm nics....

So the question could be:
how do I forbid a vm to access tagged vlans? and how do I allow some other to access them, instead?
Obviously I'm always talking about tagging vlan from guest VMs.


What about OVSPatchPort to interconnect bridges and using vlan tags on them?

Thanks
Click to expand...

You can set the vlan tag outside of the VM when you create the vNIC for the virtual machine. My guess is that if you do not specify a tag then the VM will be able to access the whole trunk and you will be able to tag inside of the vm onto all of them VLAN's.

This is exactly what you need yes?

You don't need to use the native vlan 1 for anything.
 
Yes I know that I can set the VLAN tag on the VM nic from proxmox GUI, and yes if no tag is specified the VM has full access to the network.
Hmm.. you're just saying "put your network on a different VLAN than default one, so you will be able to set the tag accordingly on per nic basis". Am I right?

That could be a good workaround, but I'll spend some time on trying to get it working (and keep playing with OVS).
I guess the OVSPatchPort could be the way, but until now I didn't find a way to get it working:

I created two bridges vmbr4095 and vmbr0 interconnected with an OVSPatchPort, but if I connect the VM to vmbr0 there's no traffic.
When I'll (ever) be able to get it working, I'll try to put some vlan tags on it to see it will work.
In addition, it also seems that those patch ports won't come up automatically if I restart network services since they neither show up when running ovs-vsctl show

Code: 
# network interface settings
auto lo
iface lo inet loopback


iface eth0 inet manual


iface eth1 inet manual


allow-vmbr4095 bond0
iface bond0 inet manual
    ovs_bonds eth0 eth1
    ovs_type OVSBond
    ovs_bridge vmbr4095
    ovs_options lacp=active bond_mode=balance-tcp


auto vmbr4095
iface vmbr4095 inet static
    address  172.16.4.30
    netmask  255.255.255.0
    gateway  172.16.4.1
    ovs_type OVSBridge
    ovs_ports bond0 patch4095


auto vmbr0
iface vmbr0 inet manual
    ovs_type OVSBridge
    ovs_ports patch0


allow-vmbr0 patch0
iface patch0 inet manual
    ovs_type OVSPatchPort
    ovs_bridge vmbr0
    ovs_patch_peer patch4095


allow-vmbr4095 patch4095
iface patch4095 inet manual
    ovs_type OVSPatchPort
    ovs_bridge vmbr4095
    ovs_patch_peer patch0

Thanks again
 
Last edited:
casalicomputers said:
Yes I know that I can set the VLAN tag on the VM nic from proxmox GUI, and yes if no tag is specified the VM has full access to the network.
Hmm.. you're just saying "put your network on a different VLAN than default one, so you will be able to set the tag accordingly on per nic basis". Am I right?

That could be a good workaround, but I'll spend some time on trying to get it working (and keep playing with OVS).
I guess the OVSPatchPort could be the way, but until now I didn't find a way to get it working:

I created two bridges vmbr4095 and vmbr0 interconnected with an OVSPatchPort, but if I connect the VM to vmbr0 there's no traffic.
When I'll (ever) be able to get it working, I'll try to put some vlan tags on it to see it will work.
In addition, it also seems that those patch ports won't come up automatically if I restart network services since they neither show up when running ovs-vsctl show

Code: 
# network interface settings
auto lo
iface lo inet loopback


iface eth0 inet manual


iface eth1 inet manual


allow-vmbr4095 bond0
iface bond0 inet manual
    ovs_bonds eth0 eth1
    ovs_type OVSBond
    ovs_bridge vmbr4095
    ovs_options lacp=active bond_mode=balance-tcp


auto vmbr4095
iface vmbr4095 inet static
    address  172.16.4.30
    netmask  255.255.255.0
    gateway  172.16.4.1
    ovs_type OVSBridge
    ovs_ports bond0 patch4095


auto vmbr0
iface vmbr0 inet manual
    ovs_type OVSBridge
    ovs_ports patch0


allow-vmbr0 patch0
iface patch0 inet manual
    ovs_type OVSPatchPort
    ovs_bridge vmbr0
    ovs_patch_peer patch4095


allow-vmbr4095 patch4095
iface patch4095 inet manual
    ovs_type OVSPatchPort
    ovs_bridge vmbr4095
    ovs_patch_peer patch0

Thanks again
Click to expand...

I would try it how I suggested and avoid using the native untagged vlan wherever possible.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back