HA Cluster on a cheap

[dwma]

Hi,
Wanted to do HA cluster cluster on a cheap with XCP-NG.
At minimum I'll need 3 "compute" nodes and one server or nas acting as a shared storage for these 3 (so basically 4 servers).

I want go cheaper approach with 3 servers, but dunno if it'll work well.

2 of them will be just a compute units, and one bigger with a lot of storage will be also cluster joined, and it'll only host TrueNAS as VM with all disks passed through, which will be shared data store via iSCSI or NFS (via 25gbe nics) to the compute nodes.

Do I need HA? Yes, I'm migrating from VMware and Dell VRTx which is a nice AIO (4 blade servers with shared storage), but since it's EOL I need something newer.

 

[leesteken]

Going for less nodes seems rather counter productive if you want to go for redundancy (and HA).
Please note that HA on Proxmox might be very different from HA on VMware or XCP-NG: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_ha_manager
Maybe this thread can also be of interest to you, even though you don't plan on using Ceph: https://forum.proxmox.com/threads/fabu-can-i-use-ceph-in-a-_very_-small-cluster.159671/

 

[spirit]

Hi,
Wanted to do HA cluster cluster on a cheap with XCP-NG.
At minimum I'll need 3 "compute" nodes and one server or nas acting as a shared storage for these 3 (so basically 4 servers).

I want go cheaper approach with 3 servers, but dunno if it'll work well.

2 of them will be just a compute units, and one bigger with a lot of storage will be also cluster joined, and it'll only host TrueNAS as VM with all disks passed through, which will be shared data store via iSCSI or NFS (via 25gbe nics) to the compute nodes.

Do I need HA? Yes, I'm migrating from VMware and Dell VRTx which is a nice AIO (4 blade servers with shared storage), but since it's EOL I need something newer.

you need HA, if you want to auto restart vms of a dead node to another node.
you can install corosync qdevice on your truenas as 3rd node.
https://pve.proxmox.com/wiki/Cluster_Manager#_corosync_external_vote_support

 

[dwma]

you need HA, if you want to auto restart vms of a dead node to another node.
you can install corosync qdevice on your truenas as 3rd node.
https://pve.proxmox.com/wiki/Cluster_Manager#_corosync_external_vote_support

Install corosync qdevice directly on TrueNAS? Or as VM / Container? If the 1st one - will it survive future upgrade to another major version?

 

[Johannes S]

Install corosync qdevice directly on TrueNAS? Or as VM / Container? If the 1st one - will it survive future upgrade to another major version?

As a container or VM, TrueNAS is designed as an appilanbce so that you can't install software directly on the OS.

 

[Johannes S]

BTW: If you don't have a shared storage or want to avoid the NAS as single-point of failure Storage replication might be another option if you can live with a minimal dataloss in case of failure:
https://pve.proxmox.com/wiki/Storage_Replication

For this to work you need a ZFS storage with the same name on both nodes. By default your vms/lxcs are synched every 15 minutes to the other node but you can change that schedule to one minute to several hours. In case of an failure you will loose all changes to the vm/lxc since the last sync.