Replication in Proxmox

[serg1990io]

Hello! I have a doubt about replication in Proxmox. I'm used to work with VMWare ESXi and Veeam Backup & Replication so I have a concept of "Replication" that, from what I understand, is different from the one I should have to understand the best practices in Proxmox.

I have two identical physical servers: with VMWare and Veeam I can schedule jobs to have on server B an exact replica of the VMs on server A. If a VM on server A goes wrong or even if the entire server A explodes, I can power on the VMs on server B and I'm good to go (obviously loosing data changed since the last successful replication job).

I understand that "Replication" in Proxmox works in a different way because it replicates just the VMs' disks: for this reason, even if I have replication enabled for VMs between server A and server B, if something goes wrong on server A I have to manually copy config files on server B before being able to start the VM. This sounds a bit silly to me because I'm replicating VMs between servers exactly because I want a solution in case server A fails.

From what I've read HA could be the solution to my problem, but I've read that for HA you need three physical servers, two are not enough.

Am I missing something? What's the best way to have a reliable protection against hardware failure (or software issues) and reduced downtime with two servers?

Replication doesn't seem like a viable option, same for HA, maybe I should just think about manual restore from backup on the second server? That doesn't sound optimal either because of downtime but maybe it's the most sensible solution.

Thank you for your support!

 

[PwrBank]

Well, you can either use Veeam again for Proxmox and do the same thing...

Or, put your machines in a cluster and enable HA - But have a third machine that's just for quorum. Then you can have your cake and eat it too. It just needs to be a device that can witness both nodes at the same time, just in case something happens to one, it can be the vote that says who is the winner to take over during an HA event.

 

[serg1990io]

Thank you for the reply! I've experimented with a new approach to the problem, building a two nodes cluster and 3 VMs (no HA):

Initial situation:
-------------------------------------------------------------
NODE-A:
-VM100
-VM101
-VM102

NODE-B:
Empty but receives replicas from NODE-A
-------------------------------------------------------------

When the first replication job is completed I manually copy VM .conf files from NODE-A (100.conf, 101.conf, 102.conf) and paste them on NODE-B (renaming them as 1100.conf, 1101.conf, 1102.conf).


Final result:
-------------------------------------------------------------
NODE-A:
-VM100
-VM101
-VM102

NODE-B:
-VM1100
-VM1101
-VM1102
-------------------------------------------------------------

If something goes wrong on NODE-A, the only thing I have to do is to start VMs on NODE-B. I've tried the concept on a real setup and it perfectly works, the only think I have to do is to keep VMs conf files in sync between the two servers (a pretty easy job considering that I've about 10 VMs that are modified very very rarely)

I wonder if this is a terrible approach or it could make sense (considering that besides of replication I always have backups managed with PBS, this would be just a quick way to restore VMs)

Any opinion would be highly apprecciated

 

[Lukas Moravek]

Hello @serg1990io

If you have another server like you mention (PBS), there is option to run with HA cluster with two nodes and use PBS as third quorum device.

https://forum.proxmox.com/threads/2-node-ha-with-external-qdevice.135429/
https://pve.proxmox.com/wiki/Cluster_Manager#_corosync_external_vote_support

Hope this option also help you.

Lukas

 

[PwrBank]

Thank you for the reply! I've experimented with a new approach to the problem, building a two nodes cluster and 3 VMs (no HA):


Any opinion would be highly apprecciated

I guess you could do that, just setup something like SyncThing or a cron job to sync config files. That might get a little messy if it's not configured right though.

 

[Johannes S]

I wonder if this is a terrible approach or it could make sense (considering that besides of replication I always have backups managed with PBS, this would be just a quick way to restore VMs)

It should work but what's the benefit compared to the regular storage replication + ha already available from Proxmox VE?

 

[serg1990io]

Hello @serg1990io

If you have another server like you mention (PBS), there is option to run with HA cluster with two nodes and use PBS as third quorum device.

https://forum.proxmox.com/threads/2-node-ha-with-external-qdevice.135429/
https://pve.proxmox.com/wiki/Cluster_Manager#_corosync_external_vote_support

Hope this option also help you.

Lukas

Hello @Lukas Moravek

Thank you for the idea! I'll look into that!

At a first sight I feel like that HA will add a bit of unnecessary complexity on top of my setup, at the end of the day I just want to have VMs ready to start on a backup server if something goes wrong on the main one. I feel like that HA with its quorum mechanism, network requirements and third machine (even if using PBS for that sounds great) looks a bit overkill but I'll definetly do some tests in that direction, thank you!

Thank you also to @Johannes S and @PwrBank

 

[VictorSTS]

When the first replication job is completed I manually copy VM .conf files from NODE-A (100.conf, 101.conf, 102.conf) and paste them on NODE-B (renaming them as 1100.conf, 1101.conf, 1102.conf).

Be very careful with this: VMs with IDs 11xx.conf will be using disks with ID 1xx: when you delete VM 11xx it will delete the replicated disk and/or potentially leaving a dangling zvol.

Don't reinvent the wheel: use HA and use PBS as QDevice. You already need a cluster for replication, and it's quorum requirements are the same with or without HA, with the added benefit that you will not have to do anything if A-server fails, HA will do it for you. If in doubt, set up a testing enviroment with VMs and practice how it works.