Limit hosts on which a VM can run

C

cyruspy

Renowned Member
Jul 2, 2013
111
8
83
Would it be possible to limit any given VM to a subset of nodes in a cluster?.

Use case:
- I have a 4 node cluster, which run Ceph as a storage layer
- I need to run VMs equivalent to 1 physical node with Windows Server
- I will license 2 nodes for Windows Server (node1 & node2)
- I want to make sure the VM never touches node 3 & node 4.
 
cyruspy said:
- I have a 4 node cluster, which run Ceph as a storage layer
- I need to run VMs equivalent to 1 physical node with Windows Server
- I will license 2 nodes for Windows Server (node1 & node2)
Click to expand...
Maybe don't cluster Proxmox and only cluster Ceph? Then use two separate single-node Proxmox for each Windows VM and a separate Proxmox cluster (with QDevice) for the other VMs.
You can use Proxmox Datacenter Manager to manage the three separate clusters from one place.
 
  • Like
Reactions: Johannes S
leesteken said:
Maybe don't cluster Proxmox and only cluster Ceph? Then use two separate single-node Proxmox for each Windows VM and a separate Proxmox cluster (with QDevice) for the other VMs.
You can use Proxmox Datacenter Manager to manage the three separate clusters from one place.
Click to expand...
Sounds more complex than it needs to be....

I can get away with just limiting where the Windows Servers are instantiated, would hate to screw up a perfectly functional PVE cluster that's running other non Windows Server workloads.
 
Last edited:
cyruspy said:
Manual operator mistake: Would require a mandatory VM to host affinity.
Click to expand...
Maybe this? Add a newly named Storage backend to only nodes 1 & 2, then add a disk (dummy) to that VM from that storage - migration will then be unsuccessful to other nodes that do not have that Storage backend.
 
Last edited:
  • Like
Reactions: waltar
I got around to testing this today. If a VM in a group is running and manually migrated to a server outside of the group, it will immediately migrate off that server, within a few seconds. (this can be done with an empty VM, no disk or OS needed)

I didn't try it not started, this time, but I did before and as I recall that does stay on the "wrong" server which means it's when HA sees the VM started that it moves it off.

And for Windows SPLA purposes, AFAIK if the VM is (stays) off no license is required for the month.
 
gfngfn256 said:
Maybe this? Add a newly named Storage backend to only nodes 1 & 2, then add a disk (dummy) to that VM from that storage - migration will then be unsuccessful to other nodes that do not have that Storage backend.
Click to expand...
Can I add another pool on the same Ceph backend maybe?.

This would solve the "must not touch an unlicensed node" requirement.
 
SteveITS said:
I got around to testing this today. If a VM in a group is running and manually migrated to a server outside of the group, it will immediately migrate off that server, within a few seconds. (this can be done with an empty VM, no disk or OS needed)

I didn't try it not started, this time, but I did before and as I recall that does stay on the "wrong" server which means it's when HA sees the VM started that it moves it off.

And for Windows SPLA purposes, AFAIK if the VM is (stays) off no license is required for the month.
Click to expand...
The issue is it must not be started on an unlicensed node.

But interesting nonetheless, it could cover a "should not" scenario without the "license disk" trick.
 
I don't use Ceph myself - but probably create a new Ceph Pool with a specific name, lets call it vm-node-lock. Then edit the /etc/pve/storage.cfg under the vm-node-lock entry by adding nodes pve1 pve2 , save & exit. (I don't know the names of your nodes 1&2 but you get the idea). This should limit that pool to these nodes. Check in GUI under the various nodes. Then add a disk of any insignificant size, which is stored on the vm-node-lock storage to the VM. You should be good.

Good luck. Report back if successful.

EDIT: Nodes selection can probably be done in GUI too.
 
Last edited:
  • Like
Reactions: SteveITS
@gfngfn256 That does work, I didn't realize Datacenter>Storage allows picking nodes like that. I was looking at the forest, I guess, and not the trees...

This can all be done in the GUI.

To recap:
  • create new RBD storage in the existing pool
    • limit to the desired nodes
  • create a VM with a 1 GB disk in the "node lock" pool
I don't know what will happen if you tell your HA Group to use only nodes 1,3,5 and your new pool is only on 1,3. Guessing HA will at some point try and fail to migrate to node 5.

Disk doesn't need to be formatted, it works with a VM with no OS.

This also works with ZFS though it of course copies the 1GB disk to the other node.

Edit: didn't try an "unreferenced" disk
 
Last edited:
You must log in or register to reply here.
Top Bottom