2+q cluster with shared/replicated storage

[4leftpaws]

I am building a cluster with 2 PVE hosts and a quorum host. Each PVE host is running local storage with hardware raid. I would like have it set up in such a way that in the event of a node failure, all VMs running on one host can be restarted on another. Ideally with clustering, but replication with a 5-10 minute rollback is acceptable. Here are the following ideas I've had and their problems:

- Replication over ZFS
Problem: ZFS and hardware raid do no go well together. It's not recommended for many reasons, including the official documentation.

- Clustered hosts with CEPH storage
Problem: It doesn't look like I can do CEPH shared storage by using the quorum host as a MON in the same way I can with PVE clustering.

- Manual replication with something like rsync
This could work, but there is no way (I know of) to automatically restart the VMs upon the loss of a host.

Let me know if there are any options given the hardware limitations.

 

[leesteken]

Maybe don't use a cluster or replication, since 2 nodes (+ QDevice) is really minimal. Maybe run a PBS in a container on each node (to quickly and regularly backup the running VMs locally) and automatically sync between the PBSs. Then you can easily (but manually) restore your VMs when you (manually) detect that there is something wrong with the other Proxmox.
It's a bit of a clutch solution but running minimal clusters is also not much fun. The upsides are that it support higher latency (and disconnects) between Proxmox nodes and you've already taken care of regular backups (now also store them off-site).

 

[UdoB]

Each PVE host is running local storage with hardware raid.

If there is any chance to flash or configure it to IT-mode I would suggest to do so. The reasoning is that you abstain from several goodies, see here: https://forum.proxmox.com/threads/f...y-a-few-disks-should-i-use-zfs-at-all.160037/