newbie questions about replication

[hspindel]

First of all, I am absolutely thrilled with Proxmox. Have set up two PVE (call them PVE1 and PVE2) and one PBS, and they all work great.

I decided to learn about replication. Was able to cluster my two PVEs. Ran into two things:

1. I expected that a VM I replicated from PVE1 to PVE2 would then show up as a VM I could start on PVE2. But it doesn't show up in the list of VMs. The disk for the replicated VM does show up in my storage on PVE2. I did read that in order to launch the VM on PVE2 that I should copy the .conf file for the VM from PVE1 to PVE2. If PVE1 is down, will that .conf file be available?
I tried to copy the .conf file, but I get a message that the copy can't be completed because the file exists (and since I'm root, I should be able to overwrite anything). The file does not exist if I use "ls -a", so I'm very puzzled by that.

2. The hardware on PVE1 and PVE2 are different, with PVE2 being less capable. So sometimes if I restore a VM that originated on PVE1 to PVE2 it won't start on PVE2 until I adjust the hardware requirements (e.g., RAM) to better match PVE2. What's the best practice for handling this? When I create a VM on PVE1, am I supposed to keep the hardware limitations of PVE2 in mind? That would mean not taking advantage of all the hardware on PVE1!

 

[LnxBil]

I expected that a VM I replicated from PVE1 to PVE2 would then show up as a VM I could start on PVE2. But it doesn't show up in the list of VMs. The disk for the replicated VM does show up in my storage on PVE2. I did read that in order to launch the VM on PVE2 that I should copy the .conf file for the VM from PVE1 to PVE2. If PVE1 is down, will that .conf file be available?
I tried to copy the .conf file, but I get a message that the copy can't be completed because the file exists (and since I'm root, I should be able to overwrite anything). The file does not exist if I use "ls -a", so I'm very puzzled by that.

copy does not work on the filesystem behind /etc/pve, but a move will do.

The hardware on PVE1 and PVE2 are different, with PVE2 being less capable. So sometimes if I restore a VM that originated on PVE1 to PVE2 it won't start on PVE2 until I adjust the hardware requirements (e.g., RAM) to better match PVE2. What's the best practice for handling this? When I create a VM on PVE1, am I supposed to keep the hardware limitations of PVE2 in mind? That would mean not taking advantage of all the hardware on PVE1!

There is no solution for this besides adding more RAM to your second node to do this. If a single VM is to big to be started on the second node, you will not have fun with this.

Keep in mind that a two-node "cluster" will not work if you loose one node, so you will not have what you're planing for. You need at least a qdevice on your PBS, to have an odd number of machines in order to be able to start the VMs if your main node fails. Maybe look into not-clustering the nodes and setup a simple replication job. You will not have problems with moving QM configs, because they're already present.

 

[hspindel]

copy does not work on the filesystem behind /etc/pve, but a move will do.

mv source/106.conf /etc/pve/qemu-server reports:

Cannot create regular file /etc/pve/qemu-server/106.conf: File exists

The file definitely doesn't exist according to ls.

There is no solution for this besides adding more RAM to your second node to do this. If a single VM is to big to be started on the second node, you will not have fun with this.

That's what I suspected. Thanks for confirming.

Keep in mind that a two-node "cluster" will not work if you loose one node, so you will not have what you're planing for. You need at least a qdevice on your PBS, to have an odd number of machines in order to be able to start the VMs if your main node fails. Maybe look into not-clustering the nodes and setup a simple replication job. You will not have problems with moving QM configs, because they're already present.

That's not what I was looking to do. I'm not looking for automatic failover. I just want to replicate a VM to a second PVE, and go and manually start the VM on the second PVE if I ever need it. I'm pretty sure I read somewhere that I needed to create a cluster in order to replicate so that's why I created one. Not using the cluster for anything else. If I try to replicate without having a cluster, replication tells me that replication needs at least two nodes.

So I ran the replication job, and the VM disk does show up in the proper storage on the second PVE, but no QM config files anywhere.



Thank you for your help.

 

[uzumo]

Isn't it normal for conf to only be enabled on one server?

I don't think the same vmid can exist on a cluster.

https://hostkey.com/blog/38-proxmox-replication/

 

[UdoB]

Cannot create regular file /etc/pve/qemu-server/106.conf: File exists

That file (that VM with the ID 106) can only exist once in the cluster.

To verify if the old one is still present: ~# find /etc/pve/nodes -name 106.conf - probably you'll find it there. Then move that one.

 

[UdoB]

Isn't it normal for conf to only be enabled on one server?

"/etc/pve" is a fuse-mounted database. Proxmox does a lot of work to make sure that it is identical on all cluster members. Basically it reflects the current state of the cluster, including all configuration.

That's also the reason why /etc/pve is only writeable as long as Quorum is reached.

Edit: you talk about two nodes. If one is down then the other one does not quorate. See https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_corosync_external_vote_support to implement a third vote!