[SOLVED] Adding node with unique vm ids to existing cluster

[Graxo]

Hi,

I searched the forums and didnt find a solution for my case. So im going to try to explain myself the best i can
I got 4 proxmox nodes, 3 at home and 1 in a datacenter. The 3 at home are in a cluster, all with unique vm ids based on the node i roll them out. i use Ansible to automate this.

The one node in a datacenter has a OPNSense vm on it that acts like a firewall to setup a site to site vpn to my home network, so that needs to be running to able to reach it. This is where the challenge comes in.
I would like to add this node to my already existing cluster and i know a node cant have vms on it to be able to enter a cluster. But there has to be a way right? All the nodes have unique vm ids and none of them overlap with other vms, so "not being able to add a node to a cluster becuase there might be some overlapping ids for vms, doesnt apply here."

Is there someone with experience for this kind a situation, or has time to help me figure it out?

Kind regards,
Robin

 

[Johannes S]

This propably won't work for for multiple reasons:

First you can't add nodes with vms or lxcs to a cluster since their config will get overwritten:

All existing configuration in /etc/pve is overwritten when joining a cluster. In particular, a joining node cannot hold any guests, since guest IDs could otherwise conflict, and the node will inherit the cluster’s storage configuration. To join a node with existing guest, as a workaround, you can create a backup of each guest (using vzdump) and restore it under a different ID after joining. If the node’s storage layout differs, you will need to re-add the node’s storages, and adapt each storage’s node restriction to reflect on which nodes the storage is actually available. https://pve.proxmox.com/wiki/Cluster_Manager#pvecm_join_node_to_cluster

Second the corosync network needs low latency thus I would expect issuses with a setup combining local and remote nodes:

The Proxmox VE cluster stack requires a reliable network with latencies under 5 milliseconds (LAN performance) between all nodes to operate stably. While on setups with a small node count a network with higher latencies may work, this is not guaranteed and gets rather unlikely with more than three nodes and latencies above around 10 ms.

The network should not be used heavily by other members, as while corosync does not uses much bandwidth it is sensitive to latency jitters; ideally corosync runs on its own physically separated network. Especially do not use a shared network for corosync and storage (except as a potential low-priority fallback in a redundant configuration).
https://pve.proxmox.com/wiki/Cluster_Manager#pvecm_cluster_network

What do you want to achieve with your cluster?
To have everything in one manaegment UI and migrate between the local and remote nodes you could use the Proxmox Datacenter Manager:
https://pve.proxmox.com/wiki/Proxmox_Datacenter_Manager_Roadmap

 

[Graxo]

This propably won't work for for multiple reasons:

First you can't add nodes with vms or lxcs to a cluster since their config will get overwritten:


Second the corosync network needs low latency thus I would expect issuses with a setup combining local and remote nodes:


What do you want to achieve with your cluster?
To have everything in one manaegment UI and migrate between the local and remote nodes you could use the Proxmox Datacenter Manager:
https://pve.proxmox.com/wiki/Proxmox_Datacenter_Manager_Roadmap

Thanks for the quick response, this is also what i found but i thought maybe someone has a (nasty) trick.
The latency would be fine, i got fiber at home and in the datacenter, according to my monitors the latency has never been more then 5. But will look into the Datacenter Manager solution.