Proxmox Cluster and Repliation

[Dexter23]

Hi all

I have a customer with two servers, one is a Dell and the other is a Supermicro and a mini pc with proxmox install but only for quorum purpose.

Both server have only two network adapter to use, one is connected to the office switch on vlan10(office) and the 2nd card is connected directly from one server to the other(replication purpose, and the mini pc fujitsu has only one gigabit network interface.

I read on the wiki of proxmox that is better to separate lan and cluster network, so i create a new vlan 20(cluster) in the office switch only for cluster traffic, then i create the cluster on the main server (Dell) and join the other 2 nodes.

After doing the step's above i create for each VM's the replication task, but my question is how to configure proxmox to use the 2nd ethernet interface connected directly beetween server Dell (pve01) and server Supermicro (pve02) for replication?

And the 2nd question are how to configure proxmox HA to use for migration only pve01 and pve02 and not use Pve03 (mini pc fujitsu)?

Thanks in advance.

 

[aaron]

You configured a different IP subnet on the directly connected network? Then you can set this as the migration network in Datacenter -> Options. It will also be used by the VM Replication feature.

The third node does not necessarily need to run Proxmox VE. You could also use the QDevice to get a third vote. If you keep a Proxmox VE on it, you can use HA groups to specify a priority for nodes. With the "restricted" checkbox, the guests will only be recovered on the selected nodes.

 

[Dexter23]

Thanks

 

[Dexter23]

Hi
I create a cluster with 3 nodes but when i create replication task i have this error:

2023-07-25 11:16:01 100-0: start replication job
2023-07-25 11:16:01 100-0: guest => VM 100, running => 2237341
2023-07-25 11:16:01 100-0: volumes => zpool01:vm-100-disk-0
2023-07-25 11:16:01 100-0: end replication job with error: command '/usr/bin/ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=pve02' root@172.29.50.20 pvecm mtunnel -migration_network 172.29.49.10/24 -get_migration_ip' failed: exit code 255

 

[aaron]

Can you ssh from one node to the other directly? ssh root@172.29.50.20 without a password prompt? Did you modify the prompt that happens when logging in?

Does the other node also have a zpool01?

 

[Dexter23]

Can you ssh from one node to the other directly? ssh root@172.29.50.20 without a password prompt? Did you modify the prompt that happens when logging in?

Does the other node also have a zpool01?

Yes i can access without password as you can see

 

[Dexter23]

Hi @aaron
What did you mean with this "Did you modify the prompt that happens when logging in?"
I dont'customize any prompt.

 

[Dexter23]

Hi i fix the problem by do this command:

ssh -o 'HostKeyAlias=pve02' root@172.29.50.20

And now replication start

 

[aaron]

What did you mean with this "Did you modify the prompt that happens when logging in?"
I dont'customize any prompt.

In the past, people have done that, and it broke the tooling relying on SSH connections. That's why I asked.

Hi i fix the problem by do this command:

Interesting, as that is used in the actual command for the replication as well, as can be seen in the log.

pvecm mtunnel -migration_network 172.29.49.10/24 -get_migration_ip' failed: exit code 255

With fresh eyes I did notice this, though. Do you have a migration network configured? (Datacenter->Options). Can all the nodes reach each other within it?

 

[Dexter23]

Hi yes i have:

172.29.49.10/24 i change the subnet in 172.29.51.10/24

Ping from pve01 to pve02:

Ping from pve02 to pve01:

 

[esi_y]

This was not troubleshot further? I am specifically interested in that exit code 255, where does that come from?

 

[Dexter23]

Hi just fixed in the answer below