Looking for OVS Configuration

JD Allen

New Member
Mar 9, 2018
2
1
1
61
After searching the forums for the last three days....and not finding a complete answer....I wanted to see if anyone else has tried to configure Proxmox with a similar configuration....and if so, can I see your /etc/network/interfaces file :)

Here a picture of what I'm looking for: ....well...no external links allowed?! So how do you post an image on here?!?

I have two Proxmox servers in a cluster. They are all syncing correctly on a different network outside of the diagram (eth1). I want to have two virtual networks (vmbr1, vmbr2) that connect from server to server (I'm guessing a GRE or VXLAN bridge for this.) Off of vmbr0 will be a OVSIntPort for each host. My VMs will run and usually connect with one or both of the "Internal" vlans/bridges, and will access a Router VM (RTR 1, connected to both vmbr0 & vmbr1) if they need "External" access. Network connected to eth0 is just connected to a simple unmanaged gigEth switch.

Eventually this will work up to more hypervisor nodes if this PoC works (And I would think a paid subscription as well)

I have to think someone has already been doing this for years, but my Google Foo must be weak, since I can't find anything :(

Thank for the Help!
JD
 
So for those who happen across this, here is what I got working:

First, I used a post from Igor Cicimov on Multi-tenant isolation in Proxmox to get me there (no external links allowed on here, otherwise I would just post the link like you do in the rest of the world). Given that my switches are basic dumb gig ethernet switches, I was focused on coming up with an MTU setting that would work. If I set MTU on phy interface/OVS Bridge/OVS Port to anything over 1500, no traffic would flow....so I'm stuck at the vlan level with an MTU in the 1400's (...still tweaking for final value.) I have one OVS Bridge between the nodes, and multiple vlans over that to simulate multiple subnets on the VMs/CTs.

Here is my /etc/network/interfaces file from one of my nodes:
Code:
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage part of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

allow-hotplug enp6s0

iface enp6s0 inet static
        address  10.1.1.20
        netmask  255.255.255.0
        gateway  10.1.1.1
        broadcast  10.1.1.255
        dns 10.1.1.28
        up ip link set enp6s0 up

allow-hotplug enp7s0

allow-vmbr0 enp7s0
iface enp7s0 inet manual
        ovs_bridge vmbr0
        ovs_type OVSPort
        up ip link set enp7s0 up
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        #post-up echo 1 > /proc/sys/net/ipv4/conf/enp7s0/proxy_arp

auto vmbr0
allow-ovs vmbr0
iface vmbr0 inet manual
        ovs_type OVSBridge
        ovs_ports enp7s0 tep0
        up ip link set vmbr0 up

allow-vmbr0 tep0
iface tep0 inet static
        ovs_bridge vmbr0
        ovs_type OVSIntPort
        address 10.1.45.20
        netmask 255.255.255.0

# Integration bridge
auto vmbr1
allow-ovs vmbr1
iface vmbr1 inet manual
        ovs_type OVSBridge
        ovs_ports vx1
        up ip link set vmbr1 up

# VXLAN Tunnel
allow-vmbr1 vx1
iface vx1 inet manual
        ovs_type OVSTunnel
        ovs_bridge vmbr1
        ovs_tunnel_type vxlan
        ovs_options trunks=11,22,33
        ovs_tunnel_options options:remote_ip=10.1.45.25 options:key=flow options:dst_port=4789

The other node is the same, with the exception of IP addresses and interface names. Once I noticed a typo in my interfaces file on one of the nodes, it all came right up :)

Next stop: Add a third node to the VxLAN config!
 
  • Like
Reactions: DerDanilo

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!