Datacenter Networking

[mednb]

Hello PVE enthusiast,
I am currently administering 4 Proxmox nodes and each one is siloed so there is no direct networking between nodes.
the network setup on every node is the same (it's the basic bridged network that i saw in every PVE setup tutorial):
These are the linux bridges that i have (vmbr0 is for the host's connection to the internet and vmbr1 is the "LAN" for my virtual machines to be able to connect to each other.

host wise i have the following interfaces :

with the following NAT rules:
post-up iptables -t nat -A POSTROUTING -s '10.254.100.0/24' -o vmbr0 MASQUERADE post-down iptables -t nat -D POSTROUTING -s '10.254.100.0/24' -o vmbr0 MASQUERADE

I am now trying to create a "hyper-converged" datacenter that allows me to have shared storage between the nodes and some sort of way to create a LAN that extends all the nodes and the vms that exist within them.
for now if i want to establish some sort of connection between two vms that reside in different nodes i have to assign public ips to the vms and i don't want to do that.
i know that VXLAN is a possible solution for my issue, but my networking skills and knowledge are still weak for me to grasp its concept.
my question is how should i tackle this issue, where do i begin and what should i know to reach my end goal?

 

[mgabriel]

Hi @mednb,

do you want to further explain what siloed means? I'm asking because you need to have a stable networking connection with low latency for your cluster traffic.

While VxLAN may be a solution, you can use any solution that connects your internal bridges of your Proxmox nodes, even a standard network link will do. You can also use a VLAN or any kind of vpn in between.

 

[mednb]

Hi @mednb,

do you want to further explain what siloed means? I'm asking because you need to have a stable networking connection with low latency for your cluster traffic.

While VxLAN may be a solution, you can use any solution that connects your internal bridges of your Proxmox nodes, even a standard network link will do. You can also use a VLAN or any kind of vpn in between.

What i meant by siloed is that there is no clustering for now, each node is alone on its own.
ps : each PVE instance is on a OVH VPS

 

[gunterwa]

Hello PVE enthusiast,
I am currently administering 4 Proxmox nodes and each one is siloed so there is no direct networking between nodes.
the network setup on every node is the same (it's the basic bridged network that i saw in every PVE setup tutorial):
These are the linux bridges that i have (vmbr0 is for the host's connection to the internet and vmbr1 is the "LAN" for my virtual machines to be able to connect to each other.
View attachment 57500
host wise i have the following interfaces :
View attachment 57502
with the following NAT rules:

post-up iptables -t nat -A POSTROUTING -s '10.254.100.0/24' -o vmbr0 MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.254.100.0/24' -o vmbr0 MASQUERADE


I am now trying to create a "hyper-converged" datacenter that allows me to have shared storage between the nodes and some sort of way to create a LAN that extends all the nodes and the vms that exist within them.
for now if i want to establish some sort of connection between two vms that reside in different nodes i have to assign public ips to the vms and i don't want to do that.
i know that VXLAN is a possible solution for my issue, but my networking skills and knowledge are still weak for me to grasp its concept.
my question is how should i tackle this issue, where do i begin and what should i know to reach my end goal?

SDN would address this issue, you can create a VLAN Zone and put all the NICs of different VMs in different nodes into the same VNet subnet. You need to have the same configuration on each node...