Looking for OVS Configuration

[JD Allen]

After searching the forums for the last three days....and not finding a complete answer....I wanted to see if anyone else has tried to configure Proxmox with a similar configuration....and if so, can I see your /etc/network/interfaces file

Here a picture of what I'm looking for: ....well...no external links allowed?! So how do you post an image on here?!?

I have two Proxmox servers in a cluster. They are all syncing correctly on a different network outside of the diagram (eth1). I want to have two virtual networks (vmbr1, vmbr2) that connect from server to server (I'm guessing a GRE or VXLAN bridge for this.) Off of vmbr0 will be a OVSIntPort for each host. My VMs will run and usually connect with one or both of the "Internal" vlans/bridges, and will access a Router VM (RTR 1, connected to both vmbr0 & vmbr1) if they need "External" access. Network connected to eth0 is just connected to a simple unmanaged gigEth switch.

Eventually this will work up to more hypervisor nodes if this PoC works (And I would think a paid subscription as well)

I have to think someone has already been doing this for years, but my Google Foo must be weak, since I can't find anything

Thank for the Help!
JD

 

[JD Allen]

So for those who happen across this, here is what I got working:

First, I used a post from Igor Cicimov on Multi-tenant isolation in Proxmox to get me there (no external links allowed on here, otherwise I would just post the link like you do in the rest of the world). Given that my switches are basic dumb gig ethernet switches, I was focused on coming up with an MTU setting that would work. If I set MTU on phy interface/OVS Bridge/OVS Port to anything over 1500, no traffic would flow....so I'm stuck at the vlan level with an MTU in the 1400's (...still tweaking for final value.) I have one OVS Bridge between the nodes, and multiple vlans over that to simulate multiple subnets on the VMs/CTs.

Here is my /etc/network/interfaces file from one of my nodes:

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage part of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

allow-hotplug enp6s0

iface enp6s0 inet static
        address  10.1.1.20
        netmask  255.255.255.0
        gateway  10.1.1.1
        broadcast  10.1.1.255
        dns 10.1.1.28
        up ip link set enp6s0 up

allow-hotplug enp7s0

allow-vmbr0 enp7s0
iface enp7s0 inet manual
        ovs_bridge vmbr0
        ovs_type OVSPort
        up ip link set enp7s0 up
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        #post-up echo 1 > /proc/sys/net/ipv4/conf/enp7s0/proxy_arp

auto vmbr0
allow-ovs vmbr0
iface vmbr0 inet manual
        ovs_type OVSBridge
        ovs_ports enp7s0 tep0
        up ip link set vmbr0 up

allow-vmbr0 tep0
iface tep0 inet static
        ovs_bridge vmbr0
        ovs_type OVSIntPort
        address 10.1.45.20
        netmask 255.255.255.0

# Integration bridge
auto vmbr1
allow-ovs vmbr1
iface vmbr1 inet manual
        ovs_type OVSBridge
        ovs_ports vx1
        up ip link set vmbr1 up

# VXLAN Tunnel
allow-vmbr1 vx1
iface vx1 inet manual
        ovs_type OVSTunnel
        ovs_bridge vmbr1
        ovs_tunnel_type vxlan
        ovs_options trunks=11,22,33
        ovs_tunnel_options options:remote_ip=10.1.45.25 options:key=flow options:dst_port=4789

The other node is the same, with the exception of IP addresses and interface names. Once I noticed a typo in my interfaces file on one of the nodes, it all came right up

Next stop: Add a third node to the VxLAN config!

 

[RonnySharma]

Thanks for sharing the nodes Allen. Appreciate your support