Proxmox HA across VPN - ZFS vs CEPHS

[vertigo262]

I' m sure this is a common question. But I haven't found much data about it

I'm interested in setting up a Proxmox project across VPN to multiple providers. Setting up the HA looks fairly simple. However is this doable across a high latency network. And also, the ramifications of what would be a better route to go. ZFS replication vs Cephs OSD

Obviously HA envirnoments across multiple locations is a much better method of keeping data fault tolerant. But this needs to be done with minimal bandwidth. And should be doable with block storage

This will be done most likely through a wireguard network, but I guess that is less relevant to the ability to transfer the block of Data, and if it's doable.. High Latency and speed fluctuationsw

 

[dakralex]

Hi!

In general, the HA Manager is currently capable of managing the high availability of resources within a cluster. The underlying key technology for a Proxmox VE cluster is corosync, which in itself needs a low latency of under 5 milliseconds [0], which is usually achievable with a separate network for the cluster. This network can be low-bandwidth as it only needs to send heartbeat messages around which are quite small, but separating it into its own network lowers its latency because of reduced interference with other senders. There are similar requirements for Ceph clusters, which also should be as low latency as possible but with a higher bandwidth of course [1].

So doing this in a single cluster is infeasible and there currently is not a solution except if you can provide very low latency over the WAN.

[0] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#pvecm_cluster_network_requirements
[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#pve_ceph_recommendation_network