Setup Proxmox Clustered VM network multiple public IP

[higi]

Hello,

my setup is 3 node cluster (hosted on OVH): (each server own datacenter, separate network, unique public IP)

Node1(Master):
Public ip range 111.111.111.1 - 10 (10IPS)
GW 111.111.111.111
DNS 1.1.1.1

Node2(Slave2):
Public ip range 222.222.222.1 - 10 (10IPS)
GW 222.222.222.222
DNS 2.2.2.2

Node3(Slave3):
Public ip range 333.333.333.1 - 10 (10IPS)
GW 333.333.333.333
DNS 3.3.3.3

I need to setup network like this, 10 VM running on node 1, each VM will have own local IP routed to public IP.

On Node 1:
VM1: /etc/network/interfaces
Local ip 10.0.0.10
GW: ?
DNS: ?
MASK:?
Proxmox bridge to??
Public ip 111.111.111.1

VM2: /etc/network/interfaces
Local ip 10.0.0.12
GW: ?
DNS: ?
MASK:?
Proxmox bridge to??
Public ip 111.111.111.2

When i migrate Node1 VM1 and VM2 to
Node2:

VM2: /etc/network/interfaces
Local ip 10.0.0.10
GW: ?
DNS: ?
MASK:?
Proxmox bridge??
Public ip 222.222.222.1

VM2: /etc/network/interfaces
Local ip 10.0.0.12
Proxmox bridge??
Public ip 222.222.222.2

What i need is, when i migrate VM1-10 from Node1 to Node2 and start them, they will get same local IP and get online on their new public IP (no interface edit after migrate).

I googled a lot, found similar issues, but couldnt get it to work
So please if anyone will be so kind to write me step by step guide including my example IPS.

Mostly what i need:
- Node1-3 Proxmox /etc/network/interfaces
- VM1-10 /etc/network/interfaces
- Some proxmox GUI configurations?

THX

 

[zarlo5899]

does each VM and 1 or 2 network bridges IE one just for LAN and one for public ips


and a DHCP server on every node should do want you want

 

[higi]

Right now i am on basic Proxmox installation. Not sure how to configure DHCP for proxmox VMs. And how to assign each VM own public IP (some kind of route/bridge?).

I need example /etc/network/interfaces for Proxmox + VMs.
VM 1 = local 192.168.0.1 >> public 48.224.153.1
VM 2 = local 192.168.0.2 >> public 48.224.153.2
VM 3 = local 192.168.0.3 >> public 48.224.153.3

THX

meybe there are another ways how to do this, i just need when i move 1 VM to another node, it will get access to internet + own public ip and i dont have to change network/interfaces.

 

[bobmc]

I think you're making this a little more complicated than it needs to be. From what you have posted, this is how I would approach it.

I would assign private (192.168.x.x) addresses to all of your VM's and install something like pfSense on one of your nodes and use NAT to map the private IP's to public addresses. As long as the private IP remains constant, moving VM's between nodes won't affect access.
Your VM's can use static IP's or you can assign static DHCP leases.

The drawback of this approach is that pfSense becomes a single point of failure but a load balanced/HA setup is pretty specialised and not for the faint hearted.

 

[higi]

i am still dont know how do i assgn public IP to VM LAN IP
I need example /etc/network/interfaces for Proxmox + VMs + GUI.
VM 1 = local 192.168.0.1 >> public 48.224.153.1
VM 2 = local 192.168.0.2 >> public 48.224.153.2
VM 3 = local 192.168.0.3 >> public 48.224.153.3

 

[zarlo5899]

make 2 network bridges one for LAN one for public


or you have to use iptables

 

[higi]

make 2 network bridges one for LAN one for public


or you have to use iptables

please give me example how to setup bridges, one LAN and for Public and how should look VM interfaces.

how do i tell VM to use LAN IP bridged to public IP.

THX i am a noob at bridges

 

[higi]

i am still out of solution, please help.
in short i need:
- One server (Proxmox, host machine)
- On that server, 5 VMs are created
- 5 public IP addresses available (one for each VM), let's say: 80.123.21.1, 80.123.21.2, 80.123.21.3, 80.123.21.4, 80.123.21.5

i need VMs interfaces to be set on local IP range, aka 10.0.0.1-5 and somehow tell proxmox to route any VMs on 10.0.0.1-5 to public IP 80.123.21.1-5. But public IP will not be saved on interface VMs, only on Proxmox side (some kind of bridge?).

 

[higi]

is there any freelancer or payed support for this task?
please

 

[zarlo5899]

https://www.proxmox.com/en/proxmox-ve/support should do it

 

[msalwe]

is there any freelancer or payed support for this task?
please

You need to create first 3 A entries like server1.example.com server2.example.com and server3.example.com with your domain registrar. In A record assign public ip of each server to the concerned domain name ie serverX.example.com this way all the servers will be accesible on the concerned public ip addresses. While creating the cluster ( you may need to recreate the cluster if you have given local ips earlier) use these public ips or domain names and create cluster. Then on each server create bridge in networking option via gui. The first bridge is vmbr0 and so on. You need to give ip to the bridge. Also give gateway there. This way when you shift machines from one server to another you just have to change the bridge and don’t need to edit network config of the VM hope it helps. PS add that many vmbrs as there are public ips. Gateway needs to be specified only once in vmbr0 for others only their ip needs to be given. Now to have one to one relation of ip and vm use the concerned bridge individually for each vm

 

[davidmgvaz]

Hi, saw this old thread and have the same issue. What was the solution? Can anybody share?

 

[davidmgvaz]

Just made a small test, a proxmox cluster with 2 nodes, in 2 datacenters and public IPs for the VM, in my case LXC containers:

Node1:
Public ip 111.111.111.1
GW 111.111.111.254

Node2:
Public ip range 222.222.222.1
GW 222.222.222.254

VM (LXC example):

net0: name=eth0,bridge=vmbr0,firewall=1,gw=111.111.111.254,hwaddr=XX:XX:XX:XX:XX:XX,ip=111.111.123.123/32,type=veth
net1: name=eth1,bridge=vmbr0,firewall=1,gw=222.222.222.254,hwaddr=XX:XX:XX:XX:XX:XX,ip=222.222.321.321/32,type=veth

Basically I set up 2 network cards, each one only works in one of the nodes.

I can migrate the VM between both nodes and always having network in the corresponding IP.

Is this a good way to go?