[SOLVED] SDN: Please help me debug my config

[Etienne Charlier]

Dear,

I'm trying to migrate to sdn in my cluster.

I'm doing it by configuring the "sdn" part connected to the "other" sfp+ ( named enp1s0d1) port of my installation

Here are my config files:

# cat /etc/network/interfaces
#... snipped
auto vmbr9999
iface vmbr9999 inet manual
    bridge-ports enp1s0d1
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 1900-2010

source /etc/network/interfaces.d/*

auto vmbr1901
iface vmbr1901 inet manual
    bridge-ports enp1s0d1.1901
    bridge-stp off
    bridge-fd 0

# ---------------------------------
# cat /etc/network/interfaces.d/sdn
auto MGT
iface MGT
    bridge_ports vmbr9999.1901
    bridge_stp off
    bridge_fd 0

#-----------------------------------
# cat /etc/pve/sdn/zones.cfg 
vlan: MGT
    bridge vmbr9999

#-----------------------------------
# cat /etc/pve/sdn/vnets.cfg 
vnet: MGT
    tag 1901
    zone MGT
    vlanaware 0

#-------------------------------------

Now the issue:

• If I connect my VM on vmbr1901 ( outside of SDN) then I can ping my default gateway.
• If I connect my VM on MGT interface ( though SDN) then traffic does not flow ( ping/dhcp...)

What am I missing ?

Thank you very much in advance
Etienne

 

[spirit]

as you use vlan-aware bridge, you shouldn't tag physical interface directly.

sdn is tagging bridge port directly, and mixed both don't known well.

instead using

auto vmbr1901
iface vmbr1901 inet manual
    bridge-ports enp1s0d1.1901
    bridge-stp off
    bridge-fd 0

you should have

auto vmbr1901
iface vmbr1901 inet manual
    bridge-ports vmbr9999.1901
    bridge-stp off
    bridge-fd 0

or if you really want to keep your current vmbr1901,
you should disable vlan-aware option in vmbr9999

 

[Etienne Charlier]

Hi !

Thanks a lot for your useful advice. vmbr1901 was just created to test my switching infra.
I removed all traces from it from all my nodes.
I needed to reboot all cluster nodes and ...!!!SUCCESS!! it works as documented!

Thanks a lot for the very useful feature and your helpfull support !

Merci beaucoup!

 

[Etienne Charlier]

Me again,

I managed to configure the "SDN" on the 2nd port of my NIC (enp1s0d1) thanks to Spirit's useful advice.
The first port ( enp1s0) is currently used for "production", but I'm trying to move all traffic off this port and move it to the 2nd NIC port

Spirit told us to not tag vlan on interface when using bridges. Is this also true when the interface is meant to be use by the host ?

I want to move the traffic on vlan 90-94 ( migration/nfs/corosync...) on the port enp1s0d1 while keeping vmbr9999 for SDN traffic

So to summarise, I want :

* Everything running on one nic port ( need to free switch for other purpose)
* only use "sdn" connected to vmbr9999 for vm
* have a few vlan connected on 2nd port ( enp1s0d1) to be used by the host ( for migration/management/corosync/nfs)

Thanks in advance for your help !


Here follow a snippet of the interfaces file showing the configuration connected to the first port

auto enp1s0.90
iface enp1s0.90 inet static
    address 172.17.90.100/24
#    pre-up ip link set enp1s0 mtu 9000

auto enp1s0.91
iface enp1s0.91 inet static
    address 172.17.91.100/24
#    pre-up ip link set enp1s0 mtu 9000

auto enp1s0.92
iface enp1s0.92 inet static
    address 172.17.92.100/24
#    pre-up ip link set enp1s0 mtu 9000

auto enp1s0.93
iface enp1s0.93 inet static
    address 172.17.93.100/24
#        pre-up ip link set enp1s0 mtu 9000

auto enp1s0.94
iface enp1s0.94 inet static
    address 172.17.94.100/24


auto vmbr0
iface vmbr0 inet manual
    bridge-ports enp1s0
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-89

 

[spirit]

Spirit told us to not tag vlan on interface when using bridges. Is this also true when the interface is meant to be use by the host ?

I said to use use vlan tag on interface, when you use a "vlan-aware" bridge only

vlan aware bridge -> put tag on vmbr....Y

non vlanware bridge -> put tag on enp....Y

 

[Etienne Charlier]

I said to use use vlan tag on interface, when you use a "vlan-aware" bridge only

vlan aware bridge -> put tag on vmbr....Y

non vlanware bridge -> put tag on enp....Y

OK thanks
What I need is
* vlan aware bridge for my vm ( with sdn)
* a few interfaces accessible to the host only ( for nfs/migration)

If I understand well I need to use vmbr9999.90 ... for my "host only" interfaces.
Is there any performance difference between vmbr9999.90 and enp1s0d2.90 ?

 

[spirit]

OK thanks
What I need is
* vlan aware bridge for my vm ( with sdn)
* a few interfaces accessible to the host only ( for nfs/migration)

sdn should work with non-vlan aware bridge too

If I understand well I need to use vmbr9999.90 ... for my "host only" interfaces.

yes

Is there any performance difference between vmbr9999.90 and enp1s0d2.90 ?

performance is the same.