Help with Replication

[deanfourie]

Hi There,

I have just setup Proxmox replication with HA, however all my existing VMs are in LVM format. As I understand it, I can only use ZFS or would only really be beneficial to use ZFS for replication. I am not

What are my options here? Can I convert my LVM virtual disks to ZFS, or do I need to completely set the VM up again from scratch?

Thanks

 

[gfngfn256]

You can easily move a VM's disk from one storage type to another:

1. Stop VM
2. Make sure you have full VM restorable backup
3. In GUI, select VM (left-pane), Hardware, Hard Disk (right-pane) & Press top drop-down button Disk Action & choose Move Storage. Then select Target Storage & Move disk

Good luck.

 

[deanfourie]

You can easily move a VM's disk from one storage type to another:

1. Stop VM
2. Make sure you have full VM restorable backup
3. In GUI, select VM (left-pane), Hardware, Hard Disk (right-pane) & Press top drop-down button Disk Action & choose Move Storage. Then select Target Storage & Move disk

Good luck.

Thanks for the reply.

Will this actually “convert” the disk to ZFS?

Also, do I need to create a new ZFS partition in order to do this? Right now I’ve been living on LVM so I’m not too familiar with this, only reason I’m changing is to accomodate for replication.

Can I remove my entire old Storage folder and recreate in ZFS?

Thanks

 

[deanfourie]

Thanks, I tried this, all went well however replication is still failing with.

2024-05-07 17:08:03 103-0: start replication job
2024-05-07 17:08:03 103-0: guest => CT 103, running => 1
2024-05-07 17:08:03 103-0: volumes => Replication:subvol-103-disk-0
2024-05-07 17:08:04 103-0: (remote_prepare_local_job) storage 'Replication' is not available on node 'pve'
2024-05-07 17:08:04 103-0: end replication job with error: command '/usr/bin/ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=pve' -o 'UserKnownHostsFile=/etc/pve/nodes/pve/ssh_known_hosts' -o 'GlobalKnownHostsFile=none' root@172.16.103.2 -- pvesr prepare-local-job 103-0 Replication:subvol-103-disk-0 --last_sync 0' failed: exit code 255

 

[gfngfn256]

IDW your storage setup, but just to clarify:

Will this actually “convert” the disk to ZFS?

The above procedure I mentioned assumes you have the correct Storage Type(s) already setup. So if you have a VM whose disk currently resides on a non-ZFS storage type, you can easily move it to ZFS storage type. Within the VM itself any architecture/FS maybe being used. This is independent to PVE's storage type.

 

[gfngfn256]

2024-05-07 17:08:04 103-0: (remote_prepare_local_job) storage 'Replication' is not available on node 'pve'

To be able to use Storage Replication, you need to have ZFS (local) also on the Target Node you are trying to replicate to. See here.

 

[deanfourie]

Does it matter if my Disk in in RAW format?

Will replication work on changed data only rather then always replicating the entire disk.

I read the the point of ZFS was to only replicate the changed data, however my DISK is in RAW format.

Thanks

 

[gfngfn256]

Does it matter if my Disk in in RAW format?

No. You should be good to go.

Will replication work on changed data only rather then always replicating the entire disk.

From the docs:

Only changes since the last replication (so-called deltas) need to be transferred if the guest is migrated to a node to which it already is replicated. This reduces the time needed significantly. The replication direction automatically switches if you migrate a guest to the replication target node.

For example: VM100 is currently on nodeA and gets replicated to nodeB. You migrate it to nodeB, so now it gets automatically replicated back from nodeB to nodeA.

If you migrate to a node where the guest is not replicated, the whole disk data must send over. After the migration, the replication job continues to replicate this guest to the configured nodes.

 

[deanfourie]

Can someone help me,

I have just noticed that my Master node is no longer the MASTER,

Any idea how this happened.

I added the slave to the master cluster,

 

[gfngfn256]

I have just noticed that my Master node is no longer the MASTER

Firstly, you should post different problems in different topic threads.

Secondly Proxmox Clustering uses a Multi-master cluster so each node can do all management tasks, as here. There is no master/slave situation.

 

[deanfourie]

Firstly, you should post different problems in different topic threads.

Secondly Proxmox Clustering uses a Multi-master cluster so each node can do all management tasks, as here. There is no master/slave situation.

I know but its migrating all my vms over to the other node in the cluster, which does not have an many resources as my primary node.

 

[gfngfn256]

I know but its migrating all my vms over to the other node in the cluster, which does not have an many resources as my primary node.

How many nodes do you have?

 

[deanfourie]

I have 2 nodes

 

[gfngfn256]

I have 2 nodes

I thought so. You should not have a 2 node cluster when using an HA cluster as you could/will encounter a split-brain situation. I expect this is what you are experiencing at present.

Read these notes carefully.

 

[deanfourie]

Yea I actually saw this shortly after posting, what a shame.

 

[gfngfn256]

which does not have an many resources as my primary node.

Note: if you want true HA, you should have adequate resources provided on both nodes. (As docs in above post).

Also as far as your 2 nodes go, you could consider using a QDevice as here to maintain Quorum.

 

[deanfourie]

Note: if you want true HA, you should have adequate resources provided on both nodes. (As docs in above post).

Also as far as your 2 nodes go, you could consider using a QDevice as here to maintain Quorum.

Yea I might look at this at a later stage,

It is for a small network, and I only want to set up like 2 of the most "important" vms and nodes for HA, that's why the resources are not quite matched as they don't really need to be for what it will be running.

Question, what determines what gets the highest vote therefor deciding which vm is master over the others?

 

[gfngfn256]

Question, what determines what gets the highest vote therefor deciding which vm is master over the others?

I guess you meant: "which node is master ......." (not VM).

Anyway, all of this is highly configurable through the ha-manager , by using Groups & Priorities. It can be done from GUI/CLI.
You'll need to read up on it. Start here.

 

[deanfourie]

I guess you meant: "which node is master ......." (not VM).

Anyway, all of this is highly configurable through the ha-manager , by using Groups & Priorities. It can be done from GUI/CLI.
You'll need to read up on it. Start here.

Ok yes, ill come back to this and do some research first. Thanks so much for the help!

 

[deanfourie]

Question, is a 2 node cluster reliable if I run a 3rd node as a QDevice?

Thanks