cluster between 2 homes NAT network

[winagain123]

Hello , everyone :

my home devices :
a. PPPOE router with 1 single public_ip_A , LAN : 192.168.1.1 , all port-forwarding forward to 192.168.1.2 by DMZ
b. machine with proxmox ve7.2 , LAN : 192.168.1.2 , NAT network for VMs : 10.10.10.x/24
create cluster successfully.

John's home devices :
a. PPPOE router with 1 single public_ip_B, LAN : 192.168.2.1 , , all port-forwarding forward to 192.168.2.2 by DMZ
b. machine with proxmox ve7.2 , LAN : 192.168.2.2 , NAT network for VMs : 10.10.10.x/24
when join cluster via GUI , it always show error :
Establishing API connection with host 'public_ip_A'
Login succeeded.
check cluster join API version
Request addition of this node ....

in My home's GUI , John's node Link 0 showing "192.168.0.2" (not the public_ip_B)

I assume this should be quite easy by DMZ , and a lot of people should have 2 home enviornment (or 2 offices) , but I've spent days (search through youtube/forum) try to connect to cluster still fail , hope anyone have a solid solution (without VPN).

 

[bobmc]

You should consider the use of a vpn to link the two offices together, if I understand your current setup as described *anyone* with your public IP can connect to your proxmox host and from there, potentially, to your entire LAN

Normally, you would use an IPSec VPN link to connect two remote sites together over the public internet.

 

[B.Otto]

What you are planning to do is called a 'stretched cluster' or 'metro cluster', and that is generally not possible with Proxmox. A Proxmox cluster needs low latency (<1ms) on its cluster network, and that is usually not possible over WAN (be it by VPN or not).

 

[winagain123]

Hi , bobmc and B.Otto , thanks for reply.

I don't want to use VPN (this will create much more overhead) as all traffic via SSH already , and anyone can protect the network easily by firewall rules.

I think "low latency" is not the major concerns , because if both PVE run with public IP , it should connect successfully , right ?

Last night , I decide test the proxmox backup server (PBS) , it use just one single TCP port (like any popular service we know , ssh,http,rdp....) and it receiving backup perfectly , but disadvantage : I have to restore the backup manually to a PVE in John's home to use the VM/CT , just double the storage space and time to restore.

I'm just very curious while PBS with one single TCP port , why not the cluster ?

 

[fabian]

low latency and link stability for the corosync network are definitely major concerns when designing a cluster, which is why you won't find any "deployed over WAN" success stories.

 

[winagain123]

fabian , thanks for you reply.

IMHO , most SME is not looking for 100% HA (whitch is quite difficult to achieve if it contains realtime database)

When PBS is working fine under single TCP port via WAN , but just missing a manual action "restore from backup" , do you know why PBS feature is not inside PVE ?

 

[B.Otto]

You are comparing two different things.

The Proxmox Backup Server (PBS) acts as an repository. The backups get 'replicated' asynchronously to the PBS. In the process of a backup the Proxmox Host sends data and the PBS recieves them. If there are temporary congestions (for whatever reason), the transfer just takes longer.

A Cluster has to make sure that

• at any point in time, the configuration on each cluster node is the same - no matter which host makes some changes. This means that every participant is always sending and recieving, comparing and adjusting.
• it is resilient against failures of hosts and (worse: ) failures of networking between the hosts. This means that a missing response (like from a network congestion) could (and should) be interpreted as a Failure scenario that can trigger it's automatic self-healing mechanisms.

There is simply no way to make a proper Cluster over a long distance or an unreliable connection. A common way to do that in different products is by making two clusters (one per site) and offer some ways to transfer data between them - like asynchronous replication. But as of now, Proxmox does not offer cluster-to-cluster features.

 

[winagain123]

B.Otto , thank you for your long explanation of the role of cluster and PBS.
I believe I need to change the plan by installing a PBS + PVE at John's home , then backup to his PBS , and re-create the VM/CT in necessary.