HELP: HA Setup/Replication

[lsmithx2]

Hello,

I have a three-node setup but it will not let me replicate any VM and HA does not seem to work well for me.

The replication seems to be a issue with how the servers are setup in a RAID on a Dell PowerEdge R620 but I would of thought that HA setup should work ok? But last time we had it happen the VM config files all copied over but all of the VM hard drive data did not and was still on the machine that went offline.

Can anyone help with this setup? or suggest what I can do to setup either one of them? HA is what I need to setup mainly to have our internal applications always online no matter what.

Thanks.

 

[fiona]

Hi,
for HA, you need shared storage or replicated storage (currently replication is only supported when using ZFS). The HA manager assumes that no local storage is used. While HA would not be completely useless in combination with local storage, in case of a node failure, you won't be able to reach the node's local storage either

See also: https://pve.proxmox.com/pve-docs/chapter-ha-manager.html

 

[lsmithx2]

Hi,
for HA, you need shared storage or replicated storage (currently replication is only supported when using ZFS). The HA manager assumes that no local storage is used. While HA would not be completely useless in combination with local storage, in case of a node failure, you won't be able to reach the node's local storage either

See also: https://pve.proxmox.com/pve-docs/chapter-ha-manager.html

Ok so your saying that for my current setup to work I would need remote zfs storage server?

And the move all VM hard drive data over to the remote storage server and then it will work? Or does all storage need to be ZFS? As currently the servers storage drives are setup on a Dell RAID controller which is formatted as LVM-thin on proxmox.

 

[bzb-rs]

For HA the local storage must be setup this way, if local storage is being utilized: the vms must be on zfs and all local zfs storage must have the same name across all the nodes.

 

[fiona]

Ok so your saying that for my current setup to work I would need remote zfs storage server?

And the move all VM hard drive data over to the remote storage server and then it will work? Or does all storage need to be ZFS? As currently the servers storage drives are setup on a Dell RAID controller which is formatted as LVM-thin on proxmox.

You either need

• shared storage accessible from all cluster nodes even when one node is down, see here for the table of storages.
• local ZFS storage on each node and replication jobs for each HA-managed guest. As @bzb-rs said, a precondition for repication is that the ZFS pool needs to have the same name on each node.

The second option can/will still lead to data since the last replication being lost if a node fails. So the replication schedule should be rather frequent.

 

[lsmithx2]

You either need

• shared storage accessible from all cluster nodes even when one node is down, see here for the table of storages.
• local ZFS storage on each node and replication jobs for each HA-managed guest. As @bzb-rs said, a precondition for repication is that the ZFS pool needs to have the same name on each node.

The second option can/will still lead to data since the last replication being lost if a node fails. So the replication schedule should be rather frequent.

Hello Fiona,

Thanks for the reply, I am now looking at getting a network storage server but I am currently having a look around are there and specific requirements that a network storage server would need to have the vm's stored on them like high read/write I presume but can you recommend the best spec server to look for I am willing to either custom build something or buy pre-built.

 

[fiona]

Hello Fiona,

Thanks for the reply, I am now looking at getting a network storage server but I am currently having a look around are there and specific requirements that a network storage server would need to have the vm's stored on them like high read/write I presume but can you recommend the best spec server to look for I am willing to either custom build something or buy pre-built.

Sorry, I'm not really a hardware person. But there should be lots of recommendations out there and maybe some other forum user will chime in.

 

[lsmithx2]

I am currently looking at something which is using NVME storage but I am unsure if it is too overkill.

3XS SER 2U28NE1

	1x 2U rackmount server based on Gigabyte R272-Z32, hot-swap 1200W Platinum PSUs, on-board Aspeed AST2500 graphics
	1x AMD EPYC™ 7262, S SP3, 7nm, Infinity / Zen 2, 8 Core, 16 Thread, 3.2GHz, 3.4GHz Turbo, 128MB, 155W, CPU, OEM
	1x 64GB (8x 8GB) ECC Reg DDR4 2933MHz
	1x 1.92TB Micron 7400 PRO M.2 (22x110) Enterprise SSD, PCIe Gen4 1x4 NVMe, 4400MB/s Read, 2000MB/s Write, 420k/81k IOPS
	4x 1.92TB Micron 7400 PRO U.3 Enterprise 2.5" SSD, PCIe Gen4 1x4 NVMe, 6600MB/s Read, 2200MB/s Write, 430k/81k IOPS
	1x This server has 2x 1GbE NICs plus a dedicated NIC for IPMI remote management. For faster Ethernet, Infiniband or OmniPath options, please contact us on 01204 474747 or email servers@scan.co.uk
	1x 3 Year Premium (Parts & Labour) Warranty - 1st Year Onsite, 2nd & 3rd Return to Base including lifetime support.

 

[bzb-rs]

Looks very similar to what we have however you might have to scale down in terms of size for os drive and make that 2 units to run a raid1 or similar, unless you plan on using this storage for putting vm and container data, make sense to have that size of a drive.
Depending on what the purpose is, sure and sure?
For instance we have a 2x 256gb ent ssd and we hardly use 30GB including the ISO and OS data.