Tailscale Site to Site - Proxmox Cluster

C

csnoopy35

New Member
Dec 20, 2024
4
0
1
I am trying to make some discoveries on anyone that has been able to get site to site vpn clusters to work. I have been able to do ip forwarding and subnet routers set up. however, there is a change in the IP address's and of the clusters that prevents them from communicating after they beginning and set up the cluster. It works then stops talking to eachtoher
 
What I could figure out:
You configured a vpn base on Tailscale.
Initially the PVE cluster is working.
But if some IPs change, not anynmore.

Questions:
The Tailscale vpn is established by separate devices?
You mentioned subnet routers, so after the Tailscale vpn there is a router and then the PVE node?
If the Tailscale vpn has some renegotiation taking place, maybe due to new ip address assignment on public side, PVE cluster nodes can't communicate with each other anymore?

Aside of these question, please evaluate the documention reagarding network requirements in a Cluster: https://pve.proxmox.com/wiki/Cluster_Manager#_cluster_network
As long as you can't guarantee the stated conditions, it will not work reliably anyway and all troubleshooting is for nothing.
 
  • Like
Reactions: Johannes S
fba said:
What I could figure out:
You configured a vpn base on Tailscale.
Initially the PVE cluster is working.
But if some IPs change, not anynmore.

Questions:
The Tailscale vpn is established by separate devices?
You mentioned subnet routers, so after the Tailscale vpn there is a router and then the PVE node?
If the Tailscale vpn has some renegotiation taking place, maybe due to new ip address assignment on public side, PVE cluster nodes can't communicate with each other anymore?

Aside of these question, please evaluate the documention reagarding network requirements in a Cluster: https://pve.proxmox.com/wiki/Cluster_Manager#_cluster_network
As long as you can't guarantee the stated conditions, it will not work reliably anyway and all troubleshooting is for nothing.
Click to expand...
The Tailscale vpn is established by separate devices?
yes I have a proxmox host running an ubuntu LXC and I have done a lot of testing to make sure communications work before connecting the clusters. Currently for the sake of ease and set up. I have virtualized Proxmox host until this process becomes easy without having to reflash a new rPoxmox host everytime.

You mentioned subnet routers, so after the Tailscale vpn there is a router and then the PVE node?

These are virtualized routers. The ubuntu LXC acts as a router/gateway. I am able to point my "Virtualized proxmox host" to the ip addres on another subnet. via "IP route add 192.168.x.x/24 via 10.x.x.x" as an idea. This works for reuqests and sending the initial connection call.

However, I need to understand what is happening to the cluster during setup. After the initial set up stage is run, i can still access, we will name AHOST and BHost, Bhost when trying to join the cluster AHOST setup. However, it will not let me sign in. As though the ip resolves, but the ability for it to think for itself and varify me is no longer possible on BHost


If the Tailscale vpn has some renegotiation taking place, maybe due to new ip address assignment on public side, PVE cluster nodes can't communicate with each other anymore?

Not sure what you mean by renegotiating taking place.
 
You wrote the following
there is a change in the IP address's and of the clusters
Click to expand...
I am just trying to understand what you mean. By creating a cluster no ip addresses are changed.

from your example I get the following understanding:
You create cluster on node A.
Node B joins the freshly created cluster.
You can reach the GUI of node B, but you can't login? Which realm did you select for login? Linux PAM must be selected and will still allow you to login with root.
 
Last edited:
I don't run Tailscale on my individual devices (NAS boxes, Proxmox boxes, etc.). I run it on my firewall/router (pfSense) device and then use the advertise routes feature to make sure my server VLAN goes out over Tailscale (and any other VLANs I want to be on the Tailscale network). Works much better that way, and if IP addresses change, so long as they are still in the same VLAN, everything still works.
 
louie1961 said:
I don't run Tailscale on my individual devices (NAS boxes, Proxmox boxes, etc.). I run it on my firewall/router (pfSense) device and then use the advertise routes feature to make sure my server VLAN goes out over Tailscale (and any other VLANs I want to be on the Tailscale network). Works much better that way, and if IP addresses change, so long as they are still in the same VLAN, everything still works.
Click to expand...
This may be exactly the way I need to set it up. Do you have any articles, or things that help you learn more about these. Otherwise any specific kinds of categories that I could search along with what you shared would really help me.
 
Tail scale and VLANs specifically as well as a bunch of other stuff, mostly centered around network stuff, but not completely
 
csnoopy35 said:
I am trying to make some discoveries on anyone that has been able to get site to site vpn clusters to work. I have been able to do ip forwarding and subnet routers set up. however, there is a change in the IP address's and of the clusters that prevents them from communicating after they beginning and set up the cluster. It works then stops talking to eachtoher
Click to expand...
Hi, how did you manage to setup the subnet routes?
I have 2 proxmox machines with tailscale installed in lxc (not the host). I have followed the instructions very carefully, but I cannot make it work. It seems that ip forwarding does not work.

EDIT: It was able to solve my problem. It was not related to ip forwarding.
 
Last edited:
nautilus7 said:
Hi, how did you manage to setup the subnet routes?
I have 2 proxmox machines with tailscale installed in lxc (not the host). I have followed the instructions very carefully, but I cannot make it work. It seems that ip forwarding does not work.

EDIT: It was able to solve my problem. It was not related to ip forwarding.
Click to expand...
how did you solve the problem?
 
Well, the main problem I was having was because I thought I wouldn't need both machines to have the static routes added to them in order to communicate. Stupid me! For example, in order for ping to work from machine A in subnet A to machine B in subnet B, both machines need to have the static routes added to them.

Other that this, I followed the instructions provided by tailscale.
https://tailscale.com/kb/1214/site-to-site
https://tailscale.com/kb/1181/firewalls
 
Clustering Proxmox via vpn is rather pointless since the cluster network needs low latencies ( under 5ms ) to work relieable, most if not all WAN Networks can't gurantee this. See also the Dokumentation for reference:
https://pve.proxmox.com/wiki/Cluster_Manager#pvecm_cluster_network
With other words: It might be fun to play around with vpn-clusters but don't expect stability with them.
What are you trying to achieve by that? For offsite-replication ( e.g. business-continuity ) pve-zsync can be used. And for live-migration and a single managment interface the Proxmox-Datacenter-Manager will work, even between different clusters or single-nodes.
 
there are freedoms within the “corosync requires low latency” mantra

i have a 5 node cluster over tailscale that works fine. i’ve live migrated a 20 terabyte virtual machine across continents within a few minutes (staging first, of course)

i also got tired of seeing a lot of people say “why? it’ll never work anyways!” - but it can when carefully considered
 
You must log in or register to reply here.
Top Bottom