[SOLVED] Replication does ... nothing?

[Razva]

Hey,

I've successfully setup two Proxmox 5 nodes and created a cluster. All machines have the same specs/drives.

Per requirements I'm using ZFS:

root@pmx1:~# zfs list
NAME               USED  AVAIL  REFER  MOUNTPOINT
rpool             5.38G   398G    96K  /rpool
rpool/ROOT        1.13G   398G    96K  /rpool/ROOT
rpool/ROOT/pve-1  1.13G   398G  1.13G  /
rpool/data          96K   398G    96K  /rpool/data
rpool/swap        4.25G   402G    64K  -

I've created the ZFS mount from the UI:

I've created a VM, which is stored on the ZFS mount I previously created:

Lastly I've created a Replication task that should run once at 5 minutes:

Well...guess what? It does nothing. Like...really nothing. There is no content on the Target ZFS directory.

I've tried to create one ZFS mount point per server and recreate the Replication job. Same result.

What am I doing wrong here? Any way to debug this?

As an added "bonus", I cannot delete the Replication task. I click on Remove -> Yes, nothing happens.

Thanks,
Razvan

 

[wolfgang]

Hi,

from the source node what do you get if you run this command?

pvesr run --id 104-0 --verbose

 

[Razva]

Hi,

from the source node what do you get if you run this command?

pvesr run --id 104-0 --verbose

Here's the result.

Please note that:
- there were no hostname or IP changes involved
- cluster was successfully created by using the standard wiki method
- "pvecm nodes" posts both nodes and "pvecm status" posts no errors
- I've changed the default SSH port, per this thread

I've already tried:
- running "pvecm updatecerts"
- adding each LAN IP + hostname on each server's /etc/hosts
- I'm currently running 5.0-31 on both machines, so I guess this patch is already applied

Later edit: changing back the SSH port from custom to default (22) solved the "Host key verification failed" error. No idea why Proxmox's "/usr/bin/ssh" command execution doesn't interprets settings from /etc/ssh/ssh_config. Any hints?

 

[Razva]

After solving the "Host key verification failed" problem the sync works between ZFS mounts.

But...is this the only result? I was expecting full mirroring of a VM from a host to another, including config files etc. Having a replicated raw image of a VM is useful, but in this case we're talking about backups, not replication.

Maybe I'm missing something?

 

[wolfgang]

I do not understand your problem, the config is stored in the cluster files system so every node has a copy.
In case of a node will fail you have to copy the config to the correct dir and your VM is up again.

 

[Razva]

I do not understand your problem, the config is stored in the cluster files system so every node has a copy.
In case of a node will fail you have to copy the config to the correct dir and your VM is up again.

Ok, I was under the impression that the replication is "full", including config files etc. As "you can fully replicate this VM to another node, and in case things go south you just push Start on the backup node". But I guess I'm wrong?

In this case can replication be combined with HA? Yes, data loss between syncs is obvious and acceptable in my use case.

 

[dcsapak]

yes but please be aware of the caveats described here:
https://pve.proxmox.com/wiki/Storage_Replication

 

[Razva]

yes but please be aware of the caveats described here:
https://pve.proxmox.com/wiki/Storage_Replication

Works like a charm, thank you!

Here's another question. I have 3 nodes. I'm replicating from node1 to node2; node3 contains other stuff. If node1 goes down, is Proxmox "smart enough" to migrate VMs from node1 just to node2, this being the single node that has node1 data replicated? Because obviously failing over from node1 to node3 will be a no-go, as node3 doesn't has any data from node1. Thank you!

 

[dcsapak]

you can define this yourself, just create a ha group with only node1 and node2 in it and give the vm in this group

 

[Razva]

you can define this yourself, just create a ha group with only node1 and node2 in it and give the vm in this group

Great stuff, thank you so much! I owe you one.

Now if only we could failover a tad faster...?