[SOLVED] Dedicated ceph storage nodes with HA stack disabled

[semira uthsala]

Hi all,

I'm testing new setup where I have 3x storage boxes(with multiples drives) and 6x compute boxes (with two drives + more RAM) all are part of same cluster.

I installed ceph on 3x storage nodes and added all the free drives from it as OSDs. ceph is up and running and I can use mounted RBD from all 9 nodes now. Ceph mons also running on same storage boxes. Live migration and everything working fine.

But after some recent incident I really don't want to enable any HA services on storage boxes. like PV-LRM/PV-CRM and I will never use those storage boxes for compute workloads. I really want watchdog service disabled on these 3x storage nodes.

But at the same time I need rest of the 6x compute nodes to do the proper HA

Is it possible to have HA enable only part of the cluster and disable HA service like LRM/CRM watchdog in some nodes ? While testing I noticed if compute node goes down pve still try to move VMs to storage boxes even with HA services are disabled. and it wont start and I cannot move those VMs back to compute nodes due to disabled services.

Is it possible to achieve this ? Separate storage nodes and separate compute nodes in the same cluster but no HA services on storage nodes

 

[t.lamprecht]

Hi,

Is it possible to have HA enable only part of the cluster and disable HA service like LRM/CRM watchdog in some nodes ?

You can use a HA group. Add only the compute-nodes and make the group restricted, so that the HA stack doesn't try to migrate to non-group members even if there's no compute node available at all.

Use that group for all HA services.

 

[semira uthsala]

Thanks.

This is what exactly I wanted. I tested this and working fine.

Is there any way to limit only to display compute nodes in VM creation wizard after doing this ?

Regards
Semira

 

[t.lamprecht]

Is there any way to limit only to display compute nodes in VM creation wizard after doing this ?

Hmm, no currently not directly. You could maybe revoke the permissions for creating VMs on the other nodes, but I did not tried if this works without unintended side effects.

 

[semira uthsala]

Thanks for the reply.

I will test revoking permission for users.

Regards
Semira