Open-vswitch troubleshooting, having trouble getting cluster-communication vlan traffic working.

surfrock66

Active Member
Feb 10, 2020
40
8
28
41
I am setting up a 2-node proxmox cluster. These are connected to my switch via a 4-port LACP bond carrying multiple vlans each of which have a /16 subnet. Vlan 2 (subnet 10.2.0.0/16) is routable and is my management network, I can get to the web interfaces on this vlan and all is well. I have vlans/subnets 3, 4, 5, and 10, with 10 being my intended cluster communication subnet (10.10.0.0/16). In the open-vswitch configuration, none of theses have defined gateways, but the good news is a vm on the 4 VLAN is good to go with communication in and out working perfectly. As far as I can tell, all is well.

...With the exception of VLAN 10. That should be a non-routable vlan for cluster communication, meaning no gateway needed. I am not able to ping in or out on this interface on either host. I tested the vlan configuration by configuring 2 ports to be port access with vlan 10 on my switch, then hooking up 2 laptops with static IP's with no gateway, and they could ping each other fine. Broadcast traffic is working. Vlan 10 is on the trunk going to both hosts, but I cannot ping either host from the laptop on VLAN 10, and the hosts cannot ping each other with "ping -I vlan10 10.10.2.10". They ping each other's 2 interface fine.

I'm stumped, and I think it's not a switch issue, but an openvswitch issue. I'm pasting my "/etc/network/interfaces" config below for review; I have commented lines where I attach and un-attach the vlan interfaces to a bridge based on things I've read, but it didn't make a difference. Any help is appreciated!

Code:
auto lo
iface lo inet loopback

auto eno1
allow-vmbr0 eno1
iface eno1 inet manual
    ovs_mtu 9000

auto eno2
allow-vmbr0 eno2
iface eno2 inet manual
    ovs_mtu 9000

auto eno3
allow-vmbr0 eno3
iface eno3 inet manual
    ovs_mtu 9000

auto eno4
allow-vmbr0 eno4
iface eno4 inet manual
    ovs_mtu 9000

auto bond0
allow-vmbr0 bond0
iface bond0 inet manual
    ovs_bridge vmbr0
    ovs_type OVSBond
    ovs_bonds eno1 eno2 eno3 eno4
    ovs_options bond_mode=balance-tcp lacp=active other_config:lacp-time=fast
    ovs_mtu 9000

allow-ovs vmbr0
iface vmbr0 inet manual
    ovs_type OVSBridge
#    ovs_ports bond0 vlan2
    ovs_ports bond0 vlan2 vlan3 vlan4 vlan10
    ovs_mtu 9000

# Server Network
auto vlan2
allow-vmbr0 vlan2
iface vlan2 inet static
    ovs_type OVSIntPort
    ovs_bridge vmbr0
    ovs_options tag=2
    address 10.2.2.10
    netmask 255.255.0.0
    gateway 10.2.1.254
    ovs_mtu 9000

# Minecraft Network; No interface needed
auto vlan3
allow-vmbr0 vlan3
iface vlan3 inet static
    ovs_type OVSIntPort
    ovs_bridge vmbr0
    ovs_options tag=3
    ovs_mtu 9000

# Client Network; No interface needed
auto vlan4
allow-vmbr0 vlan4
iface vlan4 inet static
    ovs_type OVSIntPort
    ovs_bridge vmbr0
    ovs_options tag=4
    ovs_mtu 9000

# Cluster Communication Network; Non-Routable; Needs 1500 MTU
auto vlan10
allow-vmbr0 vlan10
iface vlan10 inet static
    ovs_type OVSIntPort
    ovs_bridge vmbr0
    ovs_opions tag=10
    address 10.10.2.10
    netmask 255.255.0.0
    ovs_mtu 1500

iface enp6s0f0 inet manual
allow-vmbr1 enp6s0f0
iface enp6s0f0 inet manual
    ovs_mtu 9000

iface enp6s0f1 inet manual
allow-vmbr1 enp6s0f1
iface enp6s0f1 inet manual
    ovs_mtu 9000

auto bond1
allow-vmbr1 bond1
iface bond1 inet manual
    ovs_bridge vmbr1
    ovs_type OVSBond
    ovs_bonds enp6s0f0 enp6s0f1
    ovs_options bond_mode=balance-tcp lacp=active other_config:lacp-time=fast
    ovs_mtu 9000

allow-ovs vmbr1
iface vmbr1 inet manual
    ovs_type OVSBridge
    ovs_ports bond1 vlan7
    ovs_mtu 9000

# Storage Network
auto vlan7
allow-vmbr1 vlan7
iface vlan7 inet static
    ovs_type OVSIntPort
    ovs_bridge vmbr1
    ovs_options tag=7
    address 10.7.2.10
    netmask 255.255.0.0
    ovs_mtu 9000
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!