HA feature without shared-storage at first and preparation for future-shared-storage

[3ncoder]

Hello,

I want to set up proxmox on mutliple nodes (more than two) on Dell Servers and I need HA for some of the machines.
Currently I do not have external storage for the machines.
The servers use HW-RAID controllers that are enabled atm.
Proxmox is installed with ext4 and the machines are ext4 as well.

I have read a lot about HA requiring external storage or setting up ZFS and that ZFS is not good on top of an external RAID.

So basically my questions are
What can I do to reach the goal of HA without external storage?
When people talk about ZFS, is it promox installed in ZFS or are the VMs installed in ZFS?
Do I always loose data when there is no external storage and only storage replication?
Aren't databases corrupted then?
How often is the replication doing the sync?

In the future I will have an external storage, can I move the VMs at this point in time over to the storage or do i need to reinstall them?
What is the best way to set up proxmox and the machines for this case in the future? (partition types,...)
On what interface does proxmox access the shared storage? Is it possible to use a seperate NIC only for shared storage access and not for VM access?

 

[bbgeek17]

Hi @3ncoder , welcome to the forum.

I have read a lot about HA requiring external storage or setting up ZFS and that ZFS is not good on top of an external RAID.

There are 3 main options:
- External _SHARED_ storage (block or file)
- Local ZFS on non-raid disks with asynchronous replication
- Ceph (distributed storage) on local disks

What can I do to reach the goal of HA without external storage?

ZFS replication or Ceph

When people talk about ZFS, is it promox installed in ZFS or are the VMs installed in ZFS?

In this context, its ZFS running on hypervisor, i.e. Proxmox

Do I always loose data when there is no external storage and only storage replication?

If you are dealing with async replication and the uncontrolled failover, yes.

Aren't databases corrupted then?

Possibly, they should recover from journal/log. Its no different if you lost power on your database server and had uncommitted data. If your database cant handle that, then it cant handle async ZFS replication.
That said, I am NOT recommending using ZFS replication for databases. You should use Application level (DB) native replication.

How often is the replication doing the sync?

I believe you can reduce the window down to every 1min. However, it also depends on the change rate and link speed.

In the future I will have an external storage, can I move the VMs at this point in time over to the storage or do i need to reinstall them?

You can move.

What is the best way to set up proxmox and the machines for this case in the future? (partition types,...)

Dedicate a disk for your OS/Hypervisor with sufficient space and protection.
Dedicated a disk for your VM storage.

On what interface does proxmox access the shared storage? I

Without any additional "rules" - whichever one can route according to Linux Kernel networking.

Is it possible to use a seperate NIC only for shared storage access and not for VM access?

Yes, of course. PVE is based on Linux - you can do a lot with Linux networking.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[esi_y]

Do I always loose data when there is no external storage and only storage replication?
Aren't databases corrupted then?
How often is the replication doing the sync?

You absolutely will lose data on a regular database in such operation, Proxmox VE will not abstract you from this. PVE is not an HA solution for database deployment.

What is the best way to set up proxmox and the machines for this case in the future? (partition types,...)

Proxmox only supports replication with ZFS, so if you need that, your VM pool needs to be ZFS, everything else is your call. You can have PVE installed on e.g. regular ext4 and only use ZFS for the VMs pool.