Problem with VLAN in Proxmox 4.1

[djouniot]

Hello,

I have a strange issue with my proxmox installation.
I had install a cluster of two nodes (one IBM 3650 M2 and a Supermicro). Each one has, at least 4 NIC.
I use an cisco SG300 small business switch (managed switch).

for the two nodes :
eth0 is configured on vmbr0 bridge for adminitrative and cluster vlan. This is connected to port 1 and 2 of the SG300. work fine.
eth1 (just for one node = IBM), is for the wan access. Connected to port 19 of the SG300 (port 20 is connected to wan) : work fine

eth 4 (on ibm) and eth2 (on Supermicro) are configured with VLAN :
vmbr100 -> eth2.100 (for one node) and eth4.100 (for the other node)
vmbr110 -> eth2.110 and eth4.110

I select vlan-aware on the config (but with or without it change nothing).
For information, I have a VM with IPCOP to firewalling the solution.
My problem arrive between the two nodes on the VLAN 100 and 110 (the ports 9 and 10 of the cisco to connect eth2 and eth4 from each nodes, are configured trunk/tagged).

I install (test) two CT (LXC) one each nodes :
- test1 -> node 1 -> vmbr100 / tag 100
- test 2 -> node 2 -> vmbr100 / tag 100

routes are configured to cross by the firewall (two virtual NIC E1000, associated with vmbr100 (tag 100) and vmbr110 (tag 110))

Then the problem :
When i ping between test1 and test 2 -> ok ping pass well
When i try something else (like ssh, wget, apt-get with outside) no communication (return !) is establish withe test2. test1 work fine (it is on the same node as the firewall). but Test2 can't communicate with serveur in node1.
A tcpdump show that tcp (or icmp) packet arrive from test2 on test1 (by example, during the ssh connexion phase).

I have all destructured create only one switch (vmbr100 - > eth2 or eth4) and this time all work fine.

If someone has an idea of the problem. Perhaps, I must envisage to use Opevswitch ?

Best regards
Dominique

 

[Richard]

Perhaps, I must envisage to use Opevswitch ?

Try openvswitch - it has a better support for VLANs used with bridges.

 

[mipsH]

Proxmox VE 4.2 + OpenvSwitch on 2 nodes in cluster (+ iSCSI storage for Quorum )


Simple scenario :

eth0 connected to Switch trunk port (802.1Q).

OpenvSwitch Bridge (vmbr0) : Bridge ports (eth0).

LXC config (LXC1 , LXC2):
Network : Bridge : vmbr0
VLAN tag : 11


VM config (VM1 , VM2):
Bridge mode
Bridge : vmbr0
VLAN tag : 11
Card : VirtIO

In this scenario (which is proposed in Wiki : https://pve.proxmox.com/wiki/Open_vSwitch ).

Quote :
"

Using Open vSwitch in Proxmox
Using Open vSwitch isn't that much different than using normal linux bridges. The main difference is instead of having a bridge per vlan, you have a single bridge containing all your vlans. Then when configuring the network interface for the VM, you would select the bridge (probably the only bridge you have), and you would also enter the VLAN Tag associated with the VLAN you want your VM to be a part of. Now there is zero effort when adding or removing VLANs!

"

P.S. Firewall is not in the use.

When all VMs and LXCs are on the same host this is working.
In the case that you move one of VMs or LXCs on another node - it can not reach each other, but tcpdump see that the packet is going through the right VLAN but the receiver node is not responding if source is from another ProxmoxVE node.

Any combination works the same.


Solution

Switching back to the old model :

eth0 connected to Switch trunk port (802.1Q).

OpenvSwitch Bridge (vmbr0) : Bridge ports (eth0).
OVS Internal port , created for VLAN11,12,13,14, .. (one OVS Int port per VLAN)
--> Requires Proxmox VE server restart on adding a new OVS Int port (new VLAN)
.

VMs and LXC configuration remains the same but everything is working.


I did not test this on 3.x since there i am using old model (described in Solution), for years, without any problem.


Question
Am i missing something here, because in problematic scenario it is clear that VLAN Tag is striped well on OpenvSwitch, between nodes , through the Hardware switch device.

The same packet is visible on another side but the receiving side does not responding at all.
So this behavior is looking like VLAN Tag is not process well but it is working (at lease partially), or some kind of post processing is not done well.

Any reasonable explanation : what happens here ?.