Private LAN between two Nodes, is it possible?

[MrPowerGamerBR]

I'm not really a networking guy, so forgive me if this is a very stupid question

I have two dedicated machines (let's call them Machine1 and Machine2), both of them are running Proxmox, both of them are in the same datacenter (so the latency is very small betweem them) and they are in a cluster.

Is it possible to create a "LAN" so containers/VMs in the Machine1 can communicate with Machine2 (and vice-versa)? Or is it only possible if I host my own VPN and create a network adapter to use that VPN?

Currently what I'm doing is port forwarding (with IP filter to block every IP except mine) all the ports I need to access in every container/VM, but this is a bit cumbersome so I'm wondering if there's an alternative.

Thanks for the attention!

EDIT: It seems like SDN + VXLAN is what I need, but because it is in beta it doesn't seem to have a lot of threads here explaning how it works.

 

[ojaksch]

I'm using tinc for a very similar setup of four hosts located at different locations in europe.
Every host is reachable by the other hosts, as well as every container I'm using. Behaves like a normal network but with different IP ranges for every host/node. Setup of tinc is relatively clear but you'll have to read in how tinc works.

 

[MrPowerGamerBR]

I'm using tinc for a very similar setup of four hosts located at different locations in europe.
Every host is reachable by the other hosts, as well as every container I'm using. Behaves like a normal network but with different IP ranges for every host/node. Setup of tinc is relatively clear but you'll have to read in how tinc works.

I've also seen a suggestion to use tinc before, but for now I've decided to test out the experimental Software Defined Network feature with VXLANs just to see if it would work. (Because, if it does, then it would fit my use case!)

And... it seems to work pretty well! However I haven't tested in production yet.

What I did was:
1. Creating two Proxmox VMs so I could test them as a cluster. (yes, Proxmox inside Proxmox, it is very cool that you can run Proxmox inside a VM, and it works fine!)
2. Creating a cluster between them
3. Setting up the SDN feature by following the tutorial in the wiki
4. Created two Debian VMs
5. Created the interfaces (I needed to check the interface name with "ip a", because looks like the interface name changes)

And it works! I also tried live migrating a VM and it worked well (but I need to test with something else, for now I only tested with a simple "ping" command)

However I'm not sure what would be the drawbacks of doing this, would it work if I was using from Machine <-> Machine? (Currently I'm only testing on the same machine, but they are two different machines)

 

[Deleted member 93625]

@MrPowerGamerBR It's been a while since I tested this so I'm not sure if I remember this correctly. But using Linux bridge can achieve what you want, can't it? I've attached a file for you. Basically,

1. You create a Linux bridge with the same name (vmbr1) and bind it to a physical network interface (NIC 1) on each node.
2. Create a VM whose network is connected to vmbr1 on each.
3. Give IP address on each VM in the same subnet and test ping between them.

Note that the physical network interface on each node should be connected physically, meaning cabled directly or via a switch.

Eoin

 

[MrPowerGamerBR]

@MrPowerGamerBR It's been a while since I tested this so I'm not sure if I remember this correctly. But using Linux bridge can achieve what you want, can't it? I've attached a file for you. Basically,

1. You create a Linux bridge with the same name (vmbr1) and bind it to a physical network interface (NIC 1) on each node.
2. Create a VM whose network is connected to vmbr1 on each.
3. Give IP address on each VM in the same subnet and test ping between them.

Note that the physical network interface on each node should be connected physically, meaning cabled directly or via a switch.

Eoin

Yeah the issue is the "Note that the physical network interface on each node should be connected physically, meaning cabled directly or via a switch.", because in my case the dedicated servers, while they are in the same datacenter, they aren't directly connected between them. :/ (I just rent them from SoYouStart/OVH)

 

[Deleted member 93625]

@MrPowerGamerBR Yeah, okay. You need to lease a line between racks, meaning more expenses. Understood. Nothing I can help you there. But I believe with Linux bridge model, it would work.

Eoin

 

[pve_gio]

Hi @MrPowerGamerBR, how did you finally do this ? Could you please share with us your experience ? We have a few server from SoYouStart and use OVS Switch to cluster using public interfaces. (we try Linux bridge but not works, perhaps omit something). This (OVS Switch) work pretty good. No VPN, no lease line, just using public IP's beetween proxmox nodes. We are testing Proxmox SDN and so far 5/5.

 

[MrPowerGamerBR]

Hi @MrPowerGamerBR, how did you finally do this ? Could you please share with us your experience ? We have a few server from SoYouStart and use OVS Switch to cluster using public interfaces. (we try Linux bridge but not works, perhaps omit something). This (OVS Switch) work pretty good. No VPN, no lease line, just using public IP's beetween proxmox nodes. We are testing Proxmox SDN and so far 5/5.

What I ended up using is what I said in the third post: I used the experiemental SDN support in Proxmox with VXLAN. It works pretty well without any drawbacks and I haven't found any issues with it (and if I did find any problems, spirit helped me in the SDN thread he has here on the Proxmox forums)

 

[MasterCATZ]

https://pve.proxmox.com/pve-docs/chapter-pvesdn.html#pvesdn_setup_example_vxlan

I am blind where is all this webgui setup that the documentation talks about ?

this should do what I am after just a terrible guide to follow only "/etc/network/interfaces" are mentioned

and no mention of
/etc/pve/sdn/vnets.cfg
/etc/pve/sdn/zones.cfg
and who knows what other files need to be created
I guess Controllers + IPAMs + DNS are somewhere

https://www.youtube.com/watch?v=SxOadhSOfw4

finally found something helpfull .. even if its in german



and finally found the page with spirit that everyone was talking about , guess I will try OpenVswitch bridge tomorrow
just trying to work out how I am meant to be bonding my public and private networks

Does OpenVswitch bridge + vxlan work together ? it seems not all features supported ? so what is best bridge for vxlan?

https://docs.openvswitch.org/en/latest/faq/vxlan/

https://forum.proxmox.com/threads/o...vms-across-private-network.76866/#post-342238

https://git.proxmox.com/?p=pve-docs.git;a=blob_plain;f=vxlan-and-evpn.adoc;hb=HEAD