VxLAN setup

[Machi]

Hello,
I am exploring VxLAN setup currently.

My target is setup VxLAN between 2 Proxmox nodes, inside of VMs/CTs can communicate via VxLAN.

Following material from Igor Cicimov setup. Unfortunately it fail. Could you please advise me where gets wrong?

Following is my setup

On the node side

auto lo
iface lo inet loopback

iface ens192 inet manual

auto vmbr0
iface vmbr0 inet static
        address  103.103.103.103
        netmask  255.255.255.240
        gateway  103.103.103.1
        bridge_ports ens192
        bridge_stp off
        bridge_fd 0

allow-vmbr1 ens224
iface ens224 inet manual
        ovs_type OVSPort
        ovs_bridge vmbr1
        mtu 1546

auto vmbr1
iface vmbr1 inet manual
        ovs_type OVSBridge
        ovs_ports ens224 tep0

allow-vmbr1 tep0
iface tep0 inet static
        address  192.168.80.2
        netmask  255.255.255.0
        ovs_type OVSIntPort
        ovs_bridge vmbr1

auto vmbr2
iface vmbr2 inet manual
        ovs_type OVSBridge
        ovs_ports vx1
#Integration Bridge

allow-vmbr2 vx1
iface vx1 inet manual
        ovs_type OVSTunnel
        ovs_bridge vmbr2
        ovs_tunnel_type vxlan
        ovs_options trunks=71,72,73,74,75
        ovs_tunnel_options options:remote_ip=192.168.80.3 options:key=flow options:dst_port=4789

The opposite side of Proxmox is done as same, just IP is changed

Created a LXC container. Network interface binding to vmbr2. Following is ovs-vsctl show output

# 
ovs-vsctl show
cc660859-ae2e-4375-9ff7-423e5c42d18d
    Bridge "vmbr2"
        Port "vmbr2"
            Interface "vmbr2"
                type: internal
        Port "veth100i1"
            tag: 73
            Interface "veth100i1"
        Port "vx1"
            trunks: [71, 72, 73, 74, 75]
            Interface "vx1"
                type: vxlan
                options: {dst_port="4789", key=flow, remote_ip="192.168.80.3"}
    Bridge "vmbr1"
        Port "ens224"
            Interface "ens224"
        Port "tep0"
            Interface "tep0"
                type: internal
        Port "vmbr1"
            Interface "vmbr1"
                type: internal
    ovs_version: "2.6.2"

It can see the veth100i1 is connected to vmbr2. The opposite node doing same at container.

Unfortunately, two containers cannot connected to each others.

Could you please advise where is comes wrong?

Thanks!

 

[spirit]

Hi,
if you are interested, we are currently working on vxlan implementation with linux bridge.

doc is here:
https://git.proxmox.com/?p=pve-docs.git;a=blob_plain;f=vxlan-and-evpn.adoc;hb=HEAD

it's use the new ifupdown2 package to manage new /etc/network/interfaces syntax. (available in pvetest repo)

 

[Machi]

Hi,
Thanks for useful information. Hope it can be production shortly.

Currently I tested by manually edit using OVS command in unicast mode by following steps. Which can successful test will container communicate to each others:

# firstly prepare the physical interface with IP 192.168.100.2
# then config following
> ovs-vsctl add-br vmbr1
> ovs-vsctl add-port vmbr1 vxlan0 -- set interface vxlan0 type=vxlan option:remote_ip=192.168.100.3
> ovs-vsctl add-port vmbr1 vi0 -- set interface vi0 type=internal

# with VLAN tag
> ovs-vsctl add-port vmbr1 vi2 tag=72  -- set interface vi2 type=internal

Then, attend new network interface to container like

> pct set 100 -net1 name=eth1,bridge=vmbr1,ip=192.168.70.2/24

or with vlan tag

> pct set 101 -net1 name=eth1,bridge=vmbr1,tag=72,ip=192.168.72.3/24

The opposite node do the same. Test two container does it able to access each other.

Output from ovs-vsctl like following:

# ovs-vsctl show
f0f48b6a-52c1-4db5-9d85-429ba352396c
    Bridge "vmbr1"
        Port "vi3"
            trunks: [74, 75]
            Interface "vi3"
                type: internal
        Port "vxlan0"
            Interface "vxlan0"
                type: vxlan
                options: {remote_ip="192.168.100.3"}
        Port "veth101i1"
            tag: 72
            Interface "veth101i1"
        Port "vi2"
            tag: 72
            Interface "vi2"
                type: internal
        Port "veth100i1"
            Interface "veth100i1"
        Port "veth102i1"
            tag: 75
            Interface "veth102i1"
        Port "vmbr1"
            Interface "vmbr1"
                type: internal
        Port "vi0"
            Interface "vi0"
                type: internal
    ovs_version: "2.6.2"

However, from the Proxmox Web GUI, it is not able to see any new bridge or interface. It caused cannot allocate VxLAN bridge when config network interface to VMs/CTs.

Question:
- Could you please comment my setup?

- Why new ovs-bridge and ovs-port cannot read from Web GUI? It is because it did not config via /etc/network/interfaces?

Thanks!