Newbie cluster configuration: 2 servers without local network and separate public IP

[ATX]

Hi

Please consider the following configuration:

Server_1
- Public IP: X.X.X.X
- Local IP: 192.168.150.1 (location A)

Server_2
- Public IP: Y.Y.Y.Y
- Local IP: 192.168.1.1 (location B)

-> Server_1 and Server_2 are not connected through a local network
-> The two IP have no dedicated domain name (and so no named ssl certificated).

So far the /ect/network/interfaces is :

auto lo
iface lo inet loopback
iface eno1 inet manual
iface eno2 inet manual
iface enp0s20f0u8u2 inet manual
auto vmbr0
iface vmbr0 inet static
        address 192.168.150.1/24
        gateway 192.198.150.254
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

auto vmbr2
iface vmbr2 inet static
    address 11.11.11.254
    netmask 255.255.255.0
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up iptables -t nat -A POSTROUTING -s '11.11.11.254/24' -o vmbr0 -j MASQUERADE
    post-up   iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
    post-down iptables -t nat -D POSTROUTING -s '11.11.11.254/24' -o vmbr0 -j MASQUERADE

Could you please advice on how to create a cluster with the following configuration ?

Could this work with the current /etc/network/interfaces or do I need to add an interface that uses the public IP ?
Will the two node be able to "see each other" otherwise ?

Thanks for the help

 

[apoc]

Could you please advice on how to create a cluster with the following configuration ?

Don't do it. Don't expose your PVE host to the public internet!
You may consider to connect both sides via a Site2Site VPN but please do yourself a favor and don't expose your virtualization host to the public internet!

 

[ATX]

@tburger Thanks for your reply. So far I have firewall rule to only allow trafic from/to the two public IP, not other IPs. I am quite sure this is not ok for prod env.However this is not the case and and I just learning.

 

[apoc]

ok. At least you have taken "some" measures...
But what do you try to achieve? I don't think that the cluster was designed to run over WAN networks. I also can't see the point.
You can't have shared storage (well you can, but again it does not make sense) over such distances. So what exactly do you try to solve here?

 

[ATX]

@tburger In orange below why I would like to setup the cluster.

Grouping nodes into a cluster has the following advantages:

• Centralized, web based management
• Multi-master clusters: each node can do all management tasks
• pmxcfs: database-driven file system for storing configuration files, replicated in real-time on all nodes using corosync.
• Easy migration of virtual machines and containers between physical hosts
• Fast deployment
• Cluster-wide services like firewall and HA

 

[Dominic]

-> Server_1 and Server_2 are not connected through a local network

=> Different datacenters?

Clusters in Proxmox VE require really low latencies which you can usually only provide by a local network. Therefore, it is very likely that you will experience troubles later on.

 

[ATX]

@Dominic Yes two different datacenters.

I was doing the assumption that low latency is only required for for corosync real-time synchronisation (aka replication) and that it was possible not to use it.

 

[apoc]

• Easy migration of virtual machines and containers between physical host

How is that supposed to work? You need to migrate every single bit in this case. Which will just take forever over WAN.
I still think this is bad idea and you should re-think your setup/approach.

 

[ATX]

How is that supposed to work?

To be honnest dont know / dont have the knowledge. But, I dont have big VMs, and if it works on LAN and no one can explain/document me why it wont work on WAN (again: no replication of drive/vms will be done) ; then i'll go for the "this will work".

I "only" need is a centralized management and a way to "move" VMs from time to time between datacenters, period. Setup cannot be change (physical constraints). A VPN layer could be added, will this solve the issue: I dont know, whill this make it more complex: for sure.

If "WAN" is a proplematic keyword I could rephrase in a LAN only question:

How do you setup /etc/network/interfaces to make a cluster between two nodes that are EACH ON A DIFFERENT LOCAL NETWORK.

LAN ROUTER 0 IP:102.168.1.254, SUBNET; 192.168.1.X
- ROUTER 1 IP:192.168.1.1, SUBNET:192.168.2.X
- ROUTER 2 IP:192.168.1.2, SUBNET:192.168.3.X
Nodes:
- NODE 1 IP:102.168.2.10 (ROUTER 1 network)
- NODE 2 IP:102.168.3.10 (ROUTER 1 network)