Same VLAN for Node and some(!) VMs (bond/bridge problem)

[rbg]

hi,

I have for the external communication one bond0 with LACP configured (Cisco). bond0 is a trunk interface, with all VLANs I need. The problem is, that the external node address (for webinterface) is in the same VLAN, like I need for VMs. How should I configure Proxmox 3.2, that I can access the webinterface (vlan 555) and also configure a VM to use 555 (Web -> VM > Hardware > Network > VLAN), on the same bond0. I tested some variations, but they failed all.

The background is, that the VM must not be able to setup/configure a Vlan inside (apt-get install vlan ... and fire up ...) , for security reasons. So, in other words: The users inside the VM never sees anything else, than the configured VLAN -> no Trunk traffic inside the VM

I have a working cluster with 10 nodes (and growing), so migration must be possible.

Any suggestions?


Update

I have a new version, and it is working. Every VM which uses the same VLAN like the node, has to be drop into the vmbr1 bridge, all others uses the vmbr0. I hope, that it is ok, what I have done.

# Webinterface ueber v601
auto bond0.601
iface bond0.601 inet manual
vlan-raw-device bond0

auto vmbr0
iface vmbr0 inet manual
    bridge_ports bond0
    bridge_stp off
    bridge_fd 0

auto vmbr1
iface vmbr1 inet static
    address 1.2.3.4
    netmask 255.255.255.128
    gateway 1.2.3.254
    bridge_ports bond0.601
    bridge_stp off
    brdige_fd 0

 

[James Saint-Rossy]

I have a similar situation to yours. I simply tagged all traffic and have a separate interface for each VLAN. That way I can choose the specific VLAN for each host. Here's what my config looks like:

iface lo inet loopback
iface eth0 inet manual
iface eth1 inet manual

auto bond0
iface bond0 inet manual
        slaves eth0 eth1
        bond_miimon 100
        bond_mode 802.3ad
        bond_xmit_hash_policy layer3+4
        mtu 9000

auto vmbr1002
iface vmbr1002 inet static
        address  10.0.2.211
        netmask  255.255.255.0
        gateway  10.0.2.1
        bridge_ports bond0.1002
        bridge_stp off
        bridge_fd 0

auto vmbr1005
iface vmbr1005 inet manual
        bridge_ports bond0.1005
        bridge_stp off
        bridge_fd 0

auto vmbr1006
iface vmbr1006 inet manual
        bridge_ports bond0.1006
        bridge_stp off
        bridge_fd 0
....
...
..

 

[rbg]

hi,

your solution doesn't work for us, because of too many V-Lans we have

 

[James Saint-Rossy]

Hey rbg,

Gotcha. FWIW I recently encountered issues with having to bridges using the same VLAN. I had one bridge for the hosts IP and one bridge for the VM traffic. This causes some weird IO issues while using ceph.

Cheers, James.

 

[Mitya]

You can use single vmbr for all and set vlan inside VM interface configs.
To set up IP on vlan use interfaces named vmbrXvY.
Here's my config:

auto vmbr0
iface vmbr0 inet manual
bridge_ports eth2
bridge_stp off
bridge_fd 0


auto vmbr0v999
iface vmbr0v999 inet static
address 10.x.x.x
netmask 255.255.255.0
gateway 10.x.x.1
bridge_ports eth2.999
bridge_stp on
bridge_fd 0


auto vmbr0v998
iface vmbr0v998 inet static
address 192.168.y.y
netmask 255.255.255.0
bridge_ports eth2.998
bridge_stp on
bridge_fd 0

VLAN 999 is management, VLAN 998 is dedicated for CEPH. This method not works for OpenVZ (you cant set vid in container's interface config), only KVM.
My eth2 is 10G, but you should be able do it on top of bonding interface in same way.