Proxmox with OpenvSwitch

[LordY971]

Hi guys,

I have testing proxmox with OpenvSwitch. I have 2 pve host in cluster (IP: 192.168.1.100 and 192.168.1.101) and here is the interface configuration:

Spoiler: Interface configuration

auto lo
iface lo inet loopback

allow-ovs vmbr0
allow-vmbr0 eth0
iface eth0 inet manual
ovs_type OVSPort
ovs_bridge vmbr0
ovs_options vlan_mode=native-untagged tag=1
# Alternatively if you want to also restrict what vlans are allowed through
# you could use:
# ovs_options tag=1 vlan_mode=native-untagged trunks=10,20,30,40

allow-vmbr0 vlan1
iface vlan1 inet static
address 192.168.1.10X
netmask 255.255.255.0
gateway 192.168.1.1
ovs_type OVSIntPort
ovs_bridge vmbr0
ovs_options tag=1
ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif

iface eth1 inet manual

auto vmbr0
iface vmbr0 inet manual
ovs_type OVSBridge
ovs_ports eth0 vlan1 vlan20 vlan21
mtu 9000
# NOTE: we MUST mention eth0, vlan1, and vlan55 even though each
# of them lists ovs_bridge vmbr0! Not sure why it needs this
# kind of cross-referencing but it won't work without it!

allow-vmbr0 vlan20
iface vlan20 inet manual
ovs_type OVSIntPort
ovs_bridge vmbr0
ovs_options tag=20

allow-vmbr0 vlan21
iface vlan21 inet manual
ovs_type OVSIntPort
ovs_bridge vmbr0
ovs_options tag=21

I also ave 2 VMs for testing and actually the issue is on guest interfaces, only vmbr0 is available and vlan20 and vlan21 are not !

I want to implement with configuration for live migration so vlan20 and 21 will be both exist on the 2 pve hosts.

Here is the openvswitch result:

Spoiler: ovs-vsctl

root@pve01:~# ovs-vsctl show
d91fd416-788c-4448-8bf1-7c82fa0d2bc4
Bridge "vmbr0"
Port "eth0"
tag: 1
Interface "eth0"
Port "vlan1"
tag: 1
Interface "vlan1"
type: internal
Port "vlan21"
tag: 21
Interface "vlan21"
type: internal
Port "vmbr0"
Interface "vmbr0"
type: internal
Port "vlan20"
tag: 20
Interface "vlan20"
type: internal
ovs_version: "2.5.0"

Everything seems alright but I cannot attach vlan20 or 21 to the guest !

Spoiler: pveversion

root@pve01:~# pveversion -v
proxmox-ve: 4.2-60 (running kernel: 4.4.15-1-pve)
pve-manager: 4.2-17 (running version: 4.2-17/e1400248)
pve-kernel-4.4.15-1-pve: 4.4.15-60
lvm2: 2.02.116-pve2
corosync-pve: 2.4.0-1
libqb0: 1.0-1
pve-cluster: 4.0-43
qemu-server: 4.0-85
pve-firmware: 1.1-8
libpve-common-perl: 4.0-71
libpve-access-control: 4.0-19
libpve-storage-perl: 4.0-56
pve-libspice-server1: 0.12.8-1
vncterm: 1.2-1
pve-qemu-kvm: 2.6-1
pve-container: 1.0-72
pve-firewall: 2.0-29
pve-ha-manager: 1.0-33
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u2
lxc-pve: 2.0.3-4
lxcfs: 2.0.2-pve1
cgmanager: not correctly installed
criu: 1.6.0-1
openvswitch-switch: 2.5.0-1

Openvs-vswitch is from PVE repository:

dists/jessie/pve-no-subscription/binary-amd64/openvswitch-switch_2.3.2-3_amd64.deb

Can anyone tell me what's the issue with my configuration?

 

[JBOWER]

Most of the time I just set the vlan on the VM for the vmbr0 interface and it will get tagged for the vlan.

 

[LordY971]

That what I did but it's not working.
Any help from the support staff because the wiki is not helpful

 

[JBOWER]

That what I did but it's not working.
Any help from the support staff because the wiki is not helpful

From looking more at your config the PVE host should use an Internal port.

light reading on type/use of ovs interface types; http://blog.scottlowe.org/2013/05/15/examining-open-vswitch-traffic-patterns/

I have my hosts off for the weekend, I'll see about turning one on and sending you a config to try, i hope by sunday.

 

[LordY971]

From looking more at your config the PVE host should use an Internal port.

light reading on type/use of ovs interface types; http://blog.scottlowe.org/2013/05/15/examining-open-vswitch-traffic-patterns/

I have my hosts off for the weekend, I'll see about turning one on and sending you a config to try, i hope by sunday.

Okay, waiting for your config example

 

[huky]

my exapmle, bonding -> vmbr -> vlan

allow-vmbr1 bond1
iface bond1 inet manual
ovs_bonds eth2 eth3
ovs_type OVSBond
ovs_bridge vmbr1
ovs_options bond_mode=active-backup
pre-up ( ifconfig eth2 mtu 9000 && ifconfig eth3 mtu 9000 )
mtu 9000

auto vmbr1
iface vmbr1 inet manual
ovs_type OVSBridge
ovs_ports bond1 vlan253 vlan100

allow-vmbr1 vlan253
iface vlan253 inet static
address 172.31.253.1
netmask 255.255.255.0
gateway 172.31.253.254
ovs_type OVSIntPort
ovs_bridge vmbr1
ovs_options tag=253
ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif

allow-vmbr1 vlan100
iface vlan100 inet static
address 10.205.1.231
netmask 255.255.255.0
gateway 10.205.1.1
ovs_type OVSIntPort
ovs_bridge vmbr1
ovs_options tag=100
ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif

 

[JBOWER]

on the PVE Host

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage part of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual
mtu 9000

auto eth1
iface eth1 inet manual
mtu 9000

auto eth2
iface eth2 inet manual
mtu 9000

allow-vmbr0 bond0
iface bond0 inet manual
ovs_bonds eth0 eth1 eth2
ovs_type OVSBond
ovs_bridge vmbr0
ovs_options lacp=active bond_mode=balance-tcp

auto vmbr0
iface vmbr0 inet manual
ovs_type OVSBridge
ovs_ports bond0 core

allow-vmbr0 core
iface core inet static
address 10.5.60.20
netmask 255.255.255.0
gateway 10.5.60.1
ovs_type OVSIntPort
ovs_bridge vmbr0

To be able to use vlans your switch has to be setup to provide a trunk port, I do not know what type of switch you are useing but for cisco it would look like this ( in this setup I allow any vlan on the trunk that the switch is setup for ) :

interface Port-channel1
switchport trunk encapsulation dot1q
switchport trunk native vlan 60
switchport trunk pruning vlan none
switchport mode trunk
!
interface GigabitEthernet1/0/1
description neon
switchport trunk encapsulation dot1q
switchport trunk native vlan 60
switchport trunk pruning vlan none
switchport mode trunk
speed 1000
duplex full
channel-group 1 mode passive
spanning-tree portfast trunk
!
interface GigabitEthernet1/0/2
description neon
switchport trunk encapsulation dot1q
switchport trunk native vlan 60
switchport trunk pruning vlan none
switchport mode trunk
speed 1000
duplex full
channel-group 1 mode passive
spanning-tree portfast trunk
!
interface GigabitEthernet1/0/3
description neon
switchport trunk encapsulation dot1q
switchport trunk native vlan 60
switchport trunk pruning vlan none
switchport mode trunk
speed 1000
duplex full
channel-group 1 mode passive
spanning-tree portfast trunk

Note, i use vlan 60 as my default and not 1 like you where showing in your config.

Attached is a photo of how i then put a VM on a vlan. ( I'v found VirtIO to be the best model, if your VM will work with it ( windows is a PITA ), the same with multiqueues )