[SOLVED] Dedicated ceph storage nodes with HA stack disabled

semira uthsala

Active Member
Nov 19, 2019
43
6
28
33
Singapore
Hi all,

I'm testing new setup where I have 3x storage boxes(with multiples drives) and 6x compute boxes (with two drives + more RAM) all are part of same cluster.

I installed ceph on 3x storage nodes and added all the free drives from it as OSDs. ceph is up and running and I can use mounted RBD from all 9 nodes now. Ceph mons also running on same storage boxes. Live migration and everything working fine.

But after some recent incident I really don't want to enable any HA services on storage boxes. like PV-LRM/PV-CRM and I will never use those storage boxes for compute workloads. I really want watchdog service disabled on these 3x storage nodes.

But at the same time I need rest of the 6x compute nodes to do the proper HA

Is it possible to have HA enable only part of the cluster and disable HA service like LRM/CRM watchdog in some nodes ? While testing I noticed if compute node goes down pve still try to move VMs to storage boxes even with HA services are disabled. and it wont start and I cannot move those VMs back to compute nodes due to disabled services.

Is it possible to achieve this ? Separate storage nodes and separate compute nodes in the same cluster but no HA services on storage nodes
 
Last edited:
Hi,
Is it possible to have HA enable only part of the cluster and disable HA service like LRM/CRM watchdog in some nodes ?

You can use a HA group. Add only the compute-nodes and make the group restricted, so that the HA stack doesn't try to migrate to non-group members even if there's no compute node available at all.

Use that group for all HA services.
 
Thanks.

This is what exactly I wanted. I tested this and working fine.

Is there any way to limit only to display compute nodes in VM creation wizard after doing this ?

Regards
Semira
 
Is there any way to limit only to display compute nodes in VM creation wizard after doing this ?

Hmm, no currently not directly. You could maybe revoke the permissions for creating VMs on the other nodes, but I did not tried if this works without unintended side effects.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!