HA configuration with 2 OSD on a 3 nodes cluster

[dwalter]

Hello

Can you tell me if it's possible to do what I want to configure :
2 physical servers as 2 basic PVE nodes
1 small computer as a "third" node but without disks

I have configured Ceph with OSDs on each of the 2 physical servers and a pool with size of 2 and only 1 as minimum size (because it's not possible to run VM on the third node without disks)



I have created HA group like this



still without the third node. In my mind I would like that this third node act as a manager/monitor only of the cluster

However when I power off the first node where is my test VM, after a few moments the VM goes to the second node but it seems that the storage is "broken" because I can't browse the pool and the VM is power on but it's not working

Do you have any idea ?

Thanks a lot

 

[aaron]

a pool with size of 2 and only 1 as minimum

Do not do this if you like your data!
The chances are too high that you end up with corrupted or broken data if things go wrong.

If you can only do a 2-node cluster, either use a network share on a NAS or similar as shared storage, or, a good alternative is to use the guest replication feature of Proxmox VE.

You can create local ZFS pools on each node. Make sure they are called the same for the replication to work.

The shortest possible replication is once per minute. That is the only downside. If one node crashes, HA will recover the VM on the other node and the VM will power up with a slightly out of date disk image. How old depends on the replication interval and how long ago the last successful replication happened.

 

[dwalter]

thanks a lot Aaron for your answer. I will reconsider my project

 

[spirit]

(you need 3 monitor for the ceph cluster)

 

[dwalter]

Yes I have configured the third node as manager and monitor for the Ceph cluster. But I will check ZFS pool as mentioned by Aaron or change my third node and add disks to transform it as a real node