I am setting up a 2-node proxmox cluster. These are connected to my switch via a 4-port LACP bond carrying multiple vlans each of which have a /16 subnet. Vlan 2 (subnet 10.2.0.0/16) is routable and is my management network, I can get to the web interfaces on this vlan and all is well. I have vlans/subnets 3, 4, 5, and 10, with 10 being my intended cluster communication subnet (10.10.0.0/16). In the open-vswitch configuration, none of theses have defined gateways, but the good news is a vm on the 4 VLAN is good to go with communication in and out working perfectly. As far as I can tell, all is well.
...With the exception of VLAN 10. That should be a non-routable vlan for cluster communication, meaning no gateway needed. I am not able to ping in or out on this interface on either host. I tested the vlan configuration by configuring 2 ports to be port access with vlan 10 on my switch, then hooking up 2 laptops with static IP's with no gateway, and they could ping each other fine. Broadcast traffic is working. Vlan 10 is on the trunk going to both hosts, but I cannot ping either host from the laptop on VLAN 10, and the hosts cannot ping each other with "ping -I vlan10 10.10.2.10". They ping each other's 2 interface fine.
I'm stumped, and I think it's not a switch issue, but an openvswitch issue. I'm pasting my "/etc/network/interfaces" config below for review; I have commented lines where I attach and un-attach the vlan interfaces to a bridge based on things I've read, but it didn't make a difference. Any help is appreciated!
...With the exception of VLAN 10. That should be a non-routable vlan for cluster communication, meaning no gateway needed. I am not able to ping in or out on this interface on either host. I tested the vlan configuration by configuring 2 ports to be port access with vlan 10 on my switch, then hooking up 2 laptops with static IP's with no gateway, and they could ping each other fine. Broadcast traffic is working. Vlan 10 is on the trunk going to both hosts, but I cannot ping either host from the laptop on VLAN 10, and the hosts cannot ping each other with "ping -I vlan10 10.10.2.10". They ping each other's 2 interface fine.
I'm stumped, and I think it's not a switch issue, but an openvswitch issue. I'm pasting my "/etc/network/interfaces" config below for review; I have commented lines where I attach and un-attach the vlan interfaces to a bridge based on things I've read, but it didn't make a difference. Any help is appreciated!
Code:
auto lo
iface lo inet loopback
auto eno1
allow-vmbr0 eno1
iface eno1 inet manual
ovs_mtu 9000
auto eno2
allow-vmbr0 eno2
iface eno2 inet manual
ovs_mtu 9000
auto eno3
allow-vmbr0 eno3
iface eno3 inet manual
ovs_mtu 9000
auto eno4
allow-vmbr0 eno4
iface eno4 inet manual
ovs_mtu 9000
auto bond0
allow-vmbr0 bond0
iface bond0 inet manual
ovs_bridge vmbr0
ovs_type OVSBond
ovs_bonds eno1 eno2 eno3 eno4
ovs_options bond_mode=balance-tcp lacp=active other_config:lacp-time=fast
ovs_mtu 9000
allow-ovs vmbr0
iface vmbr0 inet manual
ovs_type OVSBridge
# ovs_ports bond0 vlan2
ovs_ports bond0 vlan2 vlan3 vlan4 vlan10
ovs_mtu 9000
# Server Network
auto vlan2
allow-vmbr0 vlan2
iface vlan2 inet static
ovs_type OVSIntPort
ovs_bridge vmbr0
ovs_options tag=2
address 10.2.2.10
netmask 255.255.0.0
gateway 10.2.1.254
ovs_mtu 9000
# Minecraft Network; No interface needed
auto vlan3
allow-vmbr0 vlan3
iface vlan3 inet static
ovs_type OVSIntPort
ovs_bridge vmbr0
ovs_options tag=3
ovs_mtu 9000
# Client Network; No interface needed
auto vlan4
allow-vmbr0 vlan4
iface vlan4 inet static
ovs_type OVSIntPort
ovs_bridge vmbr0
ovs_options tag=4
ovs_mtu 9000
# Cluster Communication Network; Non-Routable; Needs 1500 MTU
auto vlan10
allow-vmbr0 vlan10
iface vlan10 inet static
ovs_type OVSIntPort
ovs_bridge vmbr0
ovs_opions tag=10
address 10.10.2.10
netmask 255.255.0.0
ovs_mtu 1500
iface enp6s0f0 inet manual
allow-vmbr1 enp6s0f0
iface enp6s0f0 inet manual
ovs_mtu 9000
iface enp6s0f1 inet manual
allow-vmbr1 enp6s0f1
iface enp6s0f1 inet manual
ovs_mtu 9000
auto bond1
allow-vmbr1 bond1
iface bond1 inet manual
ovs_bridge vmbr1
ovs_type OVSBond
ovs_bonds enp6s0f0 enp6s0f1
ovs_options bond_mode=balance-tcp lacp=active other_config:lacp-time=fast
ovs_mtu 9000
allow-ovs vmbr1
iface vmbr1 inet manual
ovs_type OVSBridge
ovs_ports bond1 vlan7
ovs_mtu 9000
# Storage Network
auto vlan7
allow-vmbr1 vlan7
iface vlan7 inet static
ovs_type OVSIntPort
ovs_bridge vmbr1
ovs_options tag=7
address 10.7.2.10
netmask 255.255.0.0
ovs_mtu 9000