New network configuration with bridge, vlan interfaces.

[tab2600]

Here is my network configuration in case it might help others trying to do a similar thing. I read several posts about people trying to setup a management interface on a vlan without interfering with the existing Proxmox networking. I was specifically trying to run my vms on several different vlans but have a firewall vm act as a router on a stick with 802.1q packet being passed though to the guest. This was surprisingly difficult since every time I brought a vm online on vmbr0vX it would disable all traffic on vmbr0. The way around this I found was to move the vlan interfaces to the bridge and not the underlying interface. In my case it as a bond0. This is what the config looked like. The only problem I still have is the Proxmox UI network scripts create a bond0.2 interface when adding a vlan tag to the network interfaces. I am not sure how to work around this other than to create all of the br vlan interfaces ahead of time which is not very dynamic. It would be nice to be able to check a box on the main vmbr0 interface to specify it as a trunk so that vlan interface are created on the vmbr0 interface instead of the underlying physical interface. Anyone else doing something similar?

# network interface settingsauto lo
iface lo inet loopback


iface eth0 inet manual


iface eth1 inet manual


auto bond0
iface bond0 inet manual
slaves eth0 eth1
bond_miimon 100
bond_mode 802.3ad


auto vmbr0
iface vmbr0 inet manual
bridge_ports bond0
bridge_stp off
bridge_fd 0

auto vmbr0.2
iface vmbr0.2 inet manual
vlan_raw_device vmbr0


auto vmbr0v2
iface vmbr0v2 inet static
address 192.168.254.5
netmask 255.255.255.0
gateway 192.168.254.1
bridge_ports vmbr0.2
bridge_stp off
bridge_fd 0

 

[dietmar]

It would be nice to be able to check a box on the main vmbr0 interface to specify it as a trunk so that vlan interface are created on the vmbr0 interface instead of the underlying physical interface. Anyone else doing something similar?

AFAIR this does not work at all.

 

[tab2600]

AFAIR this does not work at all.

Can you explain why you think this won't work? I happen to have it setup currently and can confirm that it does work if I configure the underlying networking on the console instead of though the Proxmox interface.

 

[dietmar]

Can you explain why you think this won't work?

No. Just send me a working patch - I will test it then.

 

[tab2600]

No. Just send me a working patch - I will test it then.

I am not familiar with your source but I will give it a try. Can you point me in the direction of the network stack/ui source files in your git repository?

 

[dietmar]

I am not familiar with your source but I will give it a try. Can you point me in the direction of the network stack/ui source files in your git repository?

see http://pve.proxmox.com/wiki/Developer_Documentation

 

[tab2600]

see http://pve.proxmox.com/wiki/Developer_Documentation

Attached is a patch for Network.pm or is source data_PVE_Network.pm

I tested it on my machines and it works.

 

[tab2600]

Attached is a patch for Network.pm or is source data_PVE_Network.pm

I tested it on my machines and it works.

One thing I did find strange was when trying to spoof my external cable connection mac address on a vlan interface. It is not an issue with the bridge/vlan configuration however it is problem of spoofing a mac on a vlan interface in general.

http://forum.pfsense.org/index.php?topic=31984.0

To get around this I just added an additional network interface for my external connection and added a vlan tag to it. I am still using a vlan trunk for the internal traffic and I can added vlan interfaces to it on the fly just fine.

Other than that everything works as expected

 

[dietmar]

I tested it on my machines and it works.

You tested with a bridge that use a bond device?

 

[dietmar]

Please consult the mailing list archives from march:

http://pve.proxmox.com/pipermail/pve-devel/2012-March/thread.html

The main advantage of our approach is:

- vlan tagging can be don by hardware
- less (kernel) bugs

 

[tab2600]

Please consult the mailing list archives from march:

http://pve.proxmox.com/pipermail/pve-devel/2012-March/thread.html

The main advantage of our approach is:

- vlan tagging can be don by hardware
- less (kernel) bugs

I saw this thread.
http://pve.proxmox.com/pipermail/pve-devel/2012-March/002521.html

So what I am proposing may be a little different and yes I am using a 802.3ad Bond.

Currently
Physical --> BondX/EthX --> vmbr0 = Tagged Trunk Bridge

|​

BondX.2/EthX.2 --> vmbr0v2 = Untagged VLAN Bridge​

|​

Tap = Guest​

The problem with this is you can only use one type of bridge or the other. Ether you are going to use the tagged frames on vmbr0 in your guest or you are going to use the untagged bridge for the guest. Ideally you would want to be able to control the vlans as close to the guest as possible to provide the highest amount of flexibility for a high number of guests.

I am suggesting to just push the untagged bridged up higher in the stack.

Physical --> BondX/EthX --> vmbr0 = Tagged Trunk Bridge

|​

vmbr0.2 --> vmbr0v2 = Untagged VLAN Bridge​

|​

Tap = Guest​

This a few advantages,
1) You can now use the Tagged Trunk Bridge and Untagged VLAN bridge at the same time.
2) The Tagged Trunk can also carry PVID packets so you can use it as a management interface isolated from the guest traffic.
3) The vlan sub interface creation process is now isolated from the underlying physical network so you don't need special logic to create those interfaces.

As for bugs. If you tried before and it did not work, I suggest you try again because I have been using it without any problem. No dropped packets or hangs. Maybe somthing has been fixed in the kernel since you tried last. Without knowing the exact bugs it would be hard to say.
For the performance issues, even though relying on the hardware to do tagging may improve performance I think you loose a lot of flexibility at the guest level. Also, you are not loosing te existing ability to bridge to the physical interfaces anyway. Everything works as it did before the only different is at what level the vlan subinterfaces are created.

Also, it is worth noting that this is similar to the way VMWare works. The vmbr0 is now acting like the vswitch in VMWare ESX.

I suggest testing it with Proxmox 2.1 and let me know your results.

 

[dietmar]

I do not really understand that advantages. Maybe alexandre can comment on that

 

[tab2600]

I do not really understand that advantages. Maybe alexandre can comment on that

Can we get Alexandre to chime in on this or should I post to the mailing list? I would like to move this configuration to our dev environment at the place I work but I would like to know if this a true change of direction or if I will need to maintain my own fork of Proxmox patches?

 

[dietmar]

Can we get Alexandre to chime in on this or should I post to the mailing list?

Development is coordinated on the pve-devel mailing list. So yes, please join the list and post the issue there.