[SOLVED] Seperate VM Network on clustered nodes.

[Voyaller]

My config is this:

• 2 Clustered Nodes
• Open vSwitch
• Virtual pfSense for vlan segmentation
• Hypervisors NET: PVE #1 eno3 (vmbr1) directly connected to PVE #2 eno4 (vmbr1)
• VM Network: PVE #1 eno4 (vmbr2) directly connected to PVE #2 eno3 (vmbr2)

My goal:

I want to create a separate VM network across our nodes with Open vSwitch and pfSense VLAN's.

My Problem:

The two PVE nodes are clustered and connected via the Hypervisors NET (vmbr1) and working as intented. It's also a separate interface on pfSense.

Looks like there is no connection between the two nodes on the VM Network. pfSense has multiple VLAN's on vmbr2 in my case, they are working only on PVE #1 where pfSense is located. When i migrate a VM to PVE #2 it doesn't "see" the gateway on PVE #1.

PVE #1 /etc/network/interfaces

auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

auto eno4
iface eno4 inet manual
        ovs_type OVSPort
        ovs_bridge vmbr2

auto eno3
iface eno3 inet manual
        ovs_type OVSPort
        ovs_bridge vmbr1

auto vmbr0
iface vmbr0 inet manual
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
#Public Access

auto vmbr1
iface vmbr1 inet static
        address 10.1.10.2/28
        gateway 10.1.10.1
        ovs_type OVSBridge
        ovs_ports eno3
#Hypervisors NET

auto vmbr2
iface vmbr2 inet manual
        ovs_type OVSBridge
        ovs_ports eno4
#Internal NET

auto vmbr3
iface vmbr3 inet manual
        ovs_type OVSBridge

auto vmbr5
iface vmbr5 inet manual
        ovs_type OVSBridge

auto vmbr6
iface vmbr6 inet manual
        ovs_type OVSBridge

PVE #2 /etc/network/interfaces

auto lo
iface lo inet loopback

iface eno1 inet manual

auto eno4
iface eno4 inet manual
        ovs_type OVSPort
        ovs_bridge vmbr1

auto eno3
iface eno3 inet manual
        ovs_type OVSPort
        ovs_bridge vmbr2

auto eno2
iface eno2 inet manual

auto vmbr0
iface vmbr0 inet manual
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
#Public Access

auto vmbr1
iface vmbr1 inet static
        address 10.1.10.3/28
        gateway 10.1.10.1
        ovs_type OVSBridge
        ovs_ports eno4
#Hypervisors NET

auto vmbr2
iface vmbr2 inet manual
        ovs_type OVSBridge
        ovs_ports eno3
#Internal NET

auto vmbr3
iface vmbr3 inet manual
        ovs_type OVSBridge

auto vmbr4
iface vmbr4 inet manual
        ovs_type OVSBridge

auto vmbr5
iface vmbr5 inet manual
        ovs_type OVSBridge

 

[floh8]

why do you use extra firewall for that
proxmox can also configure vlan

 

[Voyaller]

why do you use extra firewall for that
proxmox can also configure vlan

Using Open vSwitch with a firewall you may only add one interface with vlan tag on each vm. On the other hand, using linux bridges you have to assign one interface for each vlan.

 

[floh8]

i dont think so. i can add 2 interfaces with different vlan tags - no problem

but to your config.
1. bad config, if u use different ports on each pve for the same connection
2. how looks your switch config or are the ports connect directly?

 

[Voyaller]

i dont think so. i can add 2 interfaces with different vlan tags - no problem

but to your config.
1. bad config, if u use different ports on each pve for the same connection
2. how looks your switch config or are the ports connect directly?

SDN solved the issue. Thank you.