[SOLVED] Proxmox HA without shared backend/equal servers

showiproute

Well-Known Member
Mar 11, 2020
609
31
48
35
Austria
Hello everyone,

I am not sure if it's possible or not so I raise this question:
Is it possible to have HA from a hypervisor perspective if I do not use a shared storage backend or if the hardware configuration of each server is different?
 
Hi,
you either need a shared storage or ZFS with replication enabled for all HA managed VMs. For the replicated scenario, if a node fails, you'll loose data since the last replication. How different is the hardware? Live migration between CPUs from different vendors can lead to issues. See also here for HA requirements.
 
Servus @Fabian_E I am using ZFS but replication fails all the time (I guess as the HDDs/SSDs have different names on the servers).
The hardware itself is totally different.
Server Proxmox1 = AMD Epyc 7002, only SSD storage
Server Proxmox2 = Intel E5-2637 v3, mostly 3.5 " HDDs (spinners) + 1x NVMe SSD for OS and some VM disks
 
Servus @Fabian_E I am using ZFS but replication fails all the time (I guess as the HDDs/SSDs have different names on the servers).
Sadly, replication where the storages have different names is not (yet) supported. You'd need to rename one of the pools (can be done with zpool export/import rather easily).

The hardware itself is totally different.
Server Proxmox1 = AMD Epyc 7002, only SSD storage
Server Proxmox2 = Intel E5-2637 v3, mostly 3.5 " HDDs (spinners) + 1x NVMe SSD for OS and some VM disks
Different storage devices shouldn't lead to any problems, but as already said, live migration between CPUs from different vendors won't always work (it might if you're lucky, but there's a reason we don't officially support it ;)). HA with only two servers is not possible, because you don't have quorum when the other server is down.
 
Sadly, replication where the storages have different names is not (yet) supported. You'd need to rename one of the pools (can be done with zpool export/import rather easily).


Different storage devices shouldn't lead to any problems, but as already said, live migration between CPUs from different vendors won't always work (it might if you're lucky, but there's a reason we don't officially support it ;)). HA with only two servers is not possible, because you don't have quorum when the other server is down.
Okay, so this is currently out of my scope to rename my storage different.

Regarding live migration between different CPUs: I have already recognised that as my VM stopped working and became unresponsive. I had to reset the VM to get it back working.
Maybe a solution would be to enable migration when a VM is powered off?
The current solution if VM is offline is only backup from server B and restore on server A - this is not really "sexy". :)
 
Okay, so this is currently out of my scope to rename my storage different.
There also is pve-zsync (needs to be installed manually), where the storage names don't need to be the same. But it's also less tightly integrated, so e.g. migration won't recognize that the disks are already replicated.

Regarding live migration between different CPUs: I have already recognised that as my VM stopped working and became unresponsive. I had to reset the VM to get it back working.
Maybe a solution would be to enable migration when a VM is powered off?
Yes, you should not live-migrate then. For offline migration there also is a targetstorage option, but it's not (yet) exposed in the UI.

The current solution if VM is offline is only backup from server B and restore on server A - this is not really "sexy". :)
 
There also is pve-zsync (needs to be installed manually), where the storage names don't need to be the same. But it's also less tightly integrated, so e.g. migration won't recognize that the disks are already replicated.


Yes, you should not live-migrate then. For offline migration there also is a targetstorage option, but it's not (yet) exposed in the UI.
Thanks pve-zsync sounds interessting to me.
Is it then possible to start the VM from the 2nd server with the hardware specifications of it?
 
Thanks pve-zsync sounds interessting to me.
Is it then possible to start the VM from the 2nd server with the hardware specifications of it?
That requires some manual interaction as described in the wiki article. But with two nodes, you don't have quorum when one is down. That means you can't write to /etc/pve/, start VMs etc. You can add a QDevice to assist with quorum, but maybe you don't even want a two node cluster at all (pve-zsync also works with standalone nodes).
 
That requires some manual interaction as described in the wiki article. But with two nodes, you don't have quorum when one is down. That means you can't write to /etc/pve/, start VMs etc. You can add a QDevice to assist with quorum, but maybe you don't even want a two node cluster at all (pve-zsync also works with standalone nodes).

Hi @Fabian_E, thanks for the information - I think my current setup (backup twice per day on PBS) is sufficient for today. If PVE will ever support HA with different hardware setups/storage names I will take another look at it.

Regarding HA of PVE itself: I am using a Raspberry Pi with openSuse as a Qdevice which works perfect without any hickups or other issues.
Code:
Membership information
----------------------
    Nodeid      Votes    Qdevice Name
0x00000002          1    A,V,NMW 10.0.5.1 (local)
0x00000003          1    A,V,NMW 10.0.5.2
0x00000000          1            Qdevice
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!