How to prevent node from hosting any VM/CT ?

[jeremy.lecour]

Hi, I have a PVE cluster with 6 nodes : 2 regular nodes for virtualization (using their RAM/CPU for running VM), and 4 nodes for storage only (Ceph).

I'd like to exclude the "ceph" nodes from the migration targets. I don't want any VM to run (RAM/CPU) on those nodes.

Is there a way to disable the virtualization part of a node ?

Thanks

--
JL

 

[xybo]

You don't need to disable virtualization on the Ceph nodes.

If you're using Proxmox VE 9, use a strict HA Node Affinity rule to allow the VMs to run only on your two compute nodes:

ha-manager rules add node-affinity compute-vms \
--resources vm:100,vm:101,vm:102 \
--nodes "pve01,pve02" \
--strict 1
This way, HA can migrate the VMs between pve01 and pve02, but will never place them on the Ceph nodes.

On older PVE versions, the equivalent was an HA Group with restricted

 

[UdoB]

I don't want any VM to run (RAM/CPU) on those nodes.

In the GUI you can specify this via Datacenter --> HA --> Affinitiy Rules.

You'll find a "Help" button in the dialog, pointing to some documentation.

 

[jeremy.lecour]

Thanks for the tips.

It seems that I need to define « HA Resources » in order to set a negative affinity, but that implies that I have to put all my VM in HA resources, where I have to set multiple values for re-balance, auto-start… which I'd rather not do for the moment. I' not sure that I need HA on this cluster.

I wished there was a simple way to say that a node must not play any virtualizatuion role, only storage.

 

[Johannes S]

Are the two nodes part of the ceph cluster? If not you could build two clusters and run a qdevice vm for the compute in the ceph cluster

 

[jeremy.lecour]

Are the two nodes part of the ceph cluster? If not you could build two clusters and run a qdevice vm for the compute in the ceph cluster

The 2 nodes (that can run VMs) are not part of the Ceph cluster.
When we designed the whole thing we thought that having a single PVE cluster would redure friction between the virtualization and the storage compared to having a 2 nodes cluster for virtualization (and local ZFS storage) and a 4 nodes cluster for Ceph storage. Maybe we are wrong. Our experience with this is limited.

Note: having a cluster with an even number of nodes (split in 2 locations) we already have a QDevice. This is obviously really important, but not an issue in our case.

 

[Johannes S]

OK then I would just have separate storage cluster. Existing Ceph clusters can be added as storage to ProxmoxVE after all and with such a setup no workload will ever be put on your storage nodes. One drawback I see though: Running computer and storage on the same node might help a little bit with performance but in your case any potential effect isn't propably there anyhow.
And if at any point you want to migrate workloads between the clusters this is possible with the Datacenter manager. You could also change your mind later and remerge compute and storage.

 

[jeremy.lecour]

OK then I would just have separate storage cluster. Existing Ceph clusters can be added as storage to ProxmoxVE after all and with such a setup no workload will ever be put on your storage nodes. One drawback I see though: Running computer and storage on the same node might help a little bit with performance but in your case any potential effect isn't propably there anyhow.
And if at any point you want to migrate workloads between the clusters this is possible with the Datacenter manager. You could also change your mind later and remerge compute and storage.

OK Thanks. I'll search for a procedure to split the cluster in 2.