First Cluster networking questions (OVS and Uplink)

[Afox]

Hello,

currently I have only one node with an OVS-Bridge running and a pretty complex VM-Networking-Setup with multiple VLANs connecting to that bridge. In the future I would like to setup a 3-Node-Cluster and am currently questioning myself, if it will be possible to spread the VMs on multiple Nodes while keeping this complex Networking-Setup.

I am also questioning myself how to connect the VMs on different Nodes to the outside if I only have one Uplink in my Colocation.

For any help I am very grateful.

Regards,

Afox

 

[Xahid]

You ask very specific question regarding your unique complex networking setup with no detail at all. that's not going to help you, unless its the question only for you?

 

[Afox]

Example: You have 3 nodes in a clustered state. The first node contains VM 100 and VM 200 that are connected on VLAN 3 via OVS-Bridge. Now VM 200 moves to the second node. How do I keep them connected? Is it even possible?

Regards,

Afox

 

[czechsys]

There is no sense having different network configs in cluster. So your second node will have same complex network setup with vlans configured on switchport.

 

[Afox]

what to do if VM 100 and VM 200 need to communicate over OVS-network if they are located on two different nodes?

Can you please be a little more detailed?

 

[D0peX]

There are many different ways on how to set this up. I currently have mulitple nodes with VMs. but i'm just using separate bridges to conntect a vm to the network. no 'internal' VM network. If you have other nodes they would have to go 'external' anyways, via a private 'internal' subnet/vlan to the other host. Hence i chose for 'bridge 'em directly to the switch' method.

what wou would do is create a Vlan for a VM network. have the OVS accept that vlan and patch that tho a bridge in ovs. this way you have have an internal network for VMs. When the time arises you get other nodes, you can bridge that vlan to the switch and have the other hosts tap in to that. for essentially the same 'internal' vm network.

 

[Afox]

Hence i chose for 'bridge 'em directly to the switch' method.

maybe I am wrong but as consequence this means there can only be one Subnet for this network, e.g. 192.168.2.0/24?

Problem is our example VMs have let´s say 3 NIC and all of them are in a different VLAN and Subnet.

what wou would do is create a Vlan for a VM network. have the OVS accept that vlan and patch that tho a bridge in ovs. this way you have have an internal network for VMs. When the time arises you get other nodes, you can bridge that vlan to the switch and have the other hosts tap in to that. for essentially the same 'internal' vm network.

with this solution, is it possible to keep the OVS-VLANs (let´s say 5, 10 and 30) and the different subnets of the VM-NICs?

 

[Afox]

hi again, am I wrong with my assumptions above?

 

[dfenwick]

You're complicating things. Let your hosts (cluster nodes) all be members of all of the VLANs you're interested in. They don't need to have IP addresses on those VLANs, just need to be in the broadcast domain of those VLANs. Assuming your hosts (nodes) are all connected to a switching infrastructure that is advertising those VLANs, your guest membership can slide between whatever host you want it to run on. If you keep the networking infrastructure uniform throughout the cluster, your guests become independent of the host on which they're running.

That's kind of the whole point of building a cluster.

 

[Afox]

I think what I did not yet understand is the role of hardware switch VLAN and OVS-VLAN and how the procedure of passing VLAN to the nodes is happening.
I mean you can make the switch tag traffic with only one VLAN-Tag at once and pass that to another port within the same VLAN. That´s the hardware side. But what happens with the OVS-VLAN-Tag and how to connect 2 or more OVS-Bridges. Maybe an config-example would help me understand.

 

[czechsys]

Think about OVS almost as about hardware switch. So, from HW switch you pass port with vlan(s) to OVS interface(s) (lacp) with bridge setup on Proxmox side without IP address. Now, you define on selected bridge OVS ports with IP address for Proxmox , storage etc., but you doesn't need to define anything about VMs. VMs just have their OVS vlan defined in VM network setup - it's like as dynamically adding port with allowed vlans.
Why you want connect 2 or more OVS bridges? If they are on same VLAN/subnet, they are connected via HW switch, if different VLAN/subnet, you need gateway somewhere.

 

[dfenwick]

Think about OVS almost as about hardware switch. So, from HW switch you pass port with vlan(s) to OVS interface(s) (lacp) with bridge setup on Proxmox side without IP address. Now, you define on selected bridge OVS ports with IP address for Proxmox , storage etc., but you doesn't need to define anything about VMs. VMs just have their OVS vlan defined in VM network setup - it's like as dynamically adding port with allowed vlans.
Why you want connect 2 or more OVS bridges? If they are on same VLAN/subnet, they are connected via HW switch, if different VLAN/subnet, you need gateway somewhere.

No no, not necessarily LACP. That's only if you're aggregating a pair of interfaces. You need to do VLAN trunking on the interfaces of interest between the switches and OVS. You trunk whichever VLANs you want presented between each of the switches and they become part of the broadcast domain for that VLAN. That's VTP.

 

[Eitan]

Did this question ever get resolved, be cause I have the same question and am looking for an answer.

For reference, this is my setup.
vm0 connected to vmbr0,vmbr1,vmbr2,vmbr3
vm1 -> vmbr1
vm2 -> vmbr2
vm3 -> vmbr3

I need to configure these VMs for HA, how do I ensure that when they failover to another node, that the internal networking will keep working?
Is it just a matter of creating OVS bridges with the same settings and names(i'm hoping...) or is there some other way of doing this?

P.S. Sorry for bumping an old post but this was the most recent and relevant post that I found on the topic...

 

[D0peX]

Did this question ever get resolved, be cause I have the same question and am looking for an answer.

For reference, this is my setup.
vm0 connected to vmbr0,vmbr1,vmbr2,vmbr3
vm1 -> vmbr1
vm2 -> vmbr2
vm3 -> vmbr3

I need to configure these VMs for HA, how do I ensure that when they failover to another node, that the internal networking will keep working?
Is it just a matter of creating OVS bridges with the same settings and names(i'm hoping...) or is there some other way of doing this?

P.S. Sorry for bumping an old post but this was the most recent and relevant post that I found on the topic...

For HA you want your failover hosts to be nearly identical in configuration.
If you have Host1 and Host2. Host1 has VMBR0,1 and 2. Host2 has VMBR0 and 1 (missing 2).
When you have a VM on host1, and a nic connected to VMBR2, then you cant get this VM working on host2, because the bridge is not there.

This is why I personally have 1 VMBR with vlans. I can tag the VM on a bridge network from the config. Proxmox wil generate a tap for that vm on the host it is on.

 

[Eitan]

yes, according to what you just stated you must still have the same vmbr setup (with the same name I assume), so it seems to me that you're trading the complexity of virtual networking for the complexity of vlans (yay), I personally would prefer not having to deal with vlans at all on the application layer and just deal with it on the hardware and virtual hardware layer.