Cluster nodes vm share the same network.

jgomes

New Member
Nov 1, 2021
4
0
1
41
Hi all,
I 'm new to proxmox and networking and general. I currently face a problem on my proxmox cluster.
I have a Proxmox cluster with 11 nodes, all have public Ips and all comunicate internaly with 1000mbit connection.
The internal ips are assigned to the nodes by a hosting provider, so I don't have any controll over the range and shape of the addresses.
My problem is regarding the vms that are running on a node, whem migrating to other node they lose all connectivity, (Obviasly, because the network configuration is based on the other node.)

Anyone as any sugestion how to solve the problem? Setting up multiple networks on the same interface? some sort of routing?

Best regards,
JG.
 
Can't you tell your hoster to put all the nodes on the same private subnet?
 
Hi,
yes all nodes are in the same private subnet.

all nodes have this config.

ino1 - vmbr0 : public ip
ino2 - vmbr2 . internal 1000mbit connection.

all nodes can have an ip of 10.31.6.160/27

the vmbr1 is the network for the vms.

the cluster is configured using the vmbr2.

Capturar.PNG
 
What kind of services are you running?
Depending on the services you are running you could setup something like haproxy to redirect public clients to the right server. But I guess for that you would need all VMs an all server in the same private subnet and the vmbr1 of all server would need to be bridged together so it wouldn't make a difference on which server the VM is running.
Is it not possible to use VLANs so all VMs could share the same subnet and reach eachother?
 
Hi.
I think I found the solutions.
I have activated the sdn module. And created a vxlan over the eno2 with a new subnet.

Now I can ping all vm in every node.

I sad "I Think" because I can ping but can not ssh or any other service.
 
Hi.
I think I found the solutions.
I have activated the sdn module. And created a vxlan over the eno2 with a new subnet.

Now I can ping all vm in every node.

I sad "I Think" because I can ping but can not ssh or any other service.
you need to reduce mtu to 1450 in your vms because of vxlan encapsulation.
 
Tks. It works like a charm.

I am trying all the options. I configured a vxlan and a EVPN.

If is not to much a trouble, do you know what are the diferences between them?

Best regards.
JG
 
Tks. It works like a charm.

I am trying all the options. I configured a vxlan and a EVPN.

If is not to much a trouble, do you know what are the diferences between them?

Best regards.
JG
The main difference, is that with evpn you can route between differents vxlan with an anycast gateway embedded inside proxmox.
(No need to have a central router/firewall like pfsense for example).
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!