xfrm interface for cluster join (only as a POC)

[4920441]

Hi,

I set up proxmox on a freshly installed debian 12 which works fine so far.

Furthermore I set up strongwan with xfrm interfaces on the proxmox server and a transfer network between another proxmox server.

The goal is, to join the proxmox server with the already working cluster on the "other" side of the xfrm interface transfer network.

The Problem is, that, If I try to join, I cannot excplicty choos the xfrm interface address and the join fails.

I know that setup is neither recommended nor may it have enough performnace, despite 1 Gig througput in both directions of the ipsec connection, but it should only be a POC anyway for moving vms to another host without the detour of a proxmox backupserver or something.


Does anyone has any idea how to get proxmox to know and "accept" the xfrm interface somehow?

Thanks a lot!

Cheers

4920441

 

[janus57]

Hi,

for moving vms to another host without the detour of a proxmox backupserver or something.

Why no using "qm remote-migrate" (Cf : https://pve.proxmox.com/pve-docs/qm.1.html) ?

And for cluster it's not the speed that is important but latency for corosync (should be under 10ms).

The Proxmox VE cluster stack requires a reliable network with latencies under 5milliseconds (LAN performance) between all nodes to operate stably. While onsetups with a small node count a network with higher latencies may work, thisis not guaranteed and gets rather unlikely with more than three nodes andlatencies above around 10 ms.

Cf : https://pve.proxmox.com/pve-docs/chapter-pvecm.html#pvecm_cluster_network_requirements

For the rest I cannot help (all I can say is i have tested successfully with wireguard between 3 nodes as POC).

Best regards,

 

[4920441]

thanks for the "remote-migrate"-hint. I didn't know that, or have forgotten about it (Which comes first....;-)

But the xfrm problem persists - I solved it by generating another vmbr device with another network and set up routing over the xfrm interface only.
That works fine - and, while testing - I am pretty impressed that even ceph works "okay'ish" over such a high latency connection.

Would use it for production, thats for sure, but in some (emergency-) Cases could be nice to have.

BTW: Has anyone else an Idea for the xfrm interface config to make it "proxmox-approved"? The problem is, proxmox does not know anything about that interface...... the debian system under proxmox works fine with it, though.