tailmox - in action

[willjasen]

hello all! i've been working here and there on tailmox, a means of clustering together proxmox servers via tailscale to allow for features such as staged zfs replications/migrations across geographically distant nodes.

i've been running the dev version for a while now on my own cluster, and i have recently wanted to surface data in regards to cluster performance, link latency, corosync statistics, etc.

while i'm not quite ready to ship a numbered version just yet, you can view stats about my live tailmox cluster at https://tailmox.com/. there is a corresponding way to interact with tailmox via the web from an administrative side, but those pages are only available through a tailscale service for security reasons.

tailmox garnered a decent amount of eyes, and i'm thankful in the interest there that has been.

 

[LnxBil]

Nice idea, but building a cluster on a volatile link with a third party VPN solution as an external SPOF is a very good recipe for disaster. If you leave out the clustering, it's a nice solution.

 

[willjasen]

the clustering is the point though! but tailmox is very much a research-style project to be used at one’s own risk. it’s worked okay for my single use case, in my own personal environment, for the way that i’ve needed it.

 

[LnxBil]

For your research project: maybe generalize the concept so that you can run two corosync rings over two different VPN cloud providers to get rid of the VPN SPOF. You will however run into too high delays with corosync, as they should be 5 ms or for smaller cluster at most 10 ms (source). It was not built to run corosync cluster over WANs.

 

[willjasen]

noted! i am working on being able to toggle back and forth between lan and tailscale for clustering, which could feed into the idea of separate vpn cloud providers. i've also had interest in implementing the idea with headscale, too. the main issue i have now is that i'm relying more on "tailscale serve" and "tailscale services", so i'd have to factor that in.

i wanted to post an update about the project as it's been a while since i've done major work on it. i've been toying around with this idea for a couple of years, having previously set up my own cluster over tailscale manually, but decided to make it more concrete in code for my ease of use, but also for others to ponder or try out too. i definitely think that it's not for most people and there is risk involved, but having run my own cluster this long at least gives one example where it's possible to do it - run with that as one would like.