[SOLVED] Cluster-wide Templates

[burly]

Is there any way to make templates available cluster wide? If not, is there any plan to add this enhancement?

I recognize that as long as the template is stored on shared storage that you can clone them (linked or full) to any other node in the cluster. However, the actual template conf information is only present on a single node. This presents an issue when you are in the "Server View", but this is easily worked around by either selecting the appropriate node or switching to either "Pool View" or "Folder View".

This leads to my primary issue, API access. I'm building a CI pipeline that needs to spin up VMs from templates during the pipeline. I want to be able to deploy to different nodes in the cluster based upon resource loads. Since the templates are only actually present on a single node, it doesn't appear to be possible to do this with a single node connection. Instead, you must first connect to the node that has the template, issue the clone and specify the actual target you want, then open another connection to the node it was deployed to in order to manage the lifecycle of the VM.

 

[bbgeek17]

Your question may be valid, but I think you need to clarify what exactly you are missing.

In your description you appear to be mixing GUI, API and CLI. Its true that some resource in UI are only see in a specific view. Its also true that when using CLI tools, such as "qm", you need to be on correct node to issue a clone.
However, if you are using API then you can connect to any node in the cluster. You do need to build the API path properly to issue the clone, but you don't need to reconnect for it.

With the way API schema is structured you just have to find the parent node first. There is no way around it. For example, you can issue "cluster/resources" to find the information you need:

pvesh get /cluster/resources --output-format json|vm_name=$1 jq -r '.[]|select(.name==env.vm_name)|.vmid')
pvesh get /cluster/resources --output-format json|vm_id=$1 jq -r '.[]|select(.vmid== (env.vm_id|tonumber) )|.node')
pvesh delete /nodes/$node/$type/$vmid

Commands above will search for ID of the a VM with given name, then extract its parent node from the json object. You can then issue clone, or delete as shown above.

In short, if you are building automation expect to issue information gathering API calls prior to action.

Good luck

---

Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[burly]

What I am missing is the templates being available on All nodes, just like shared storage targets are available on all nodes. I'm mixing GUI, API, and CLI to demonstrate the affect of the underlying lack of presence of the templates on all cluster nodes is masked from the UX in different ways. From the API, where I'm making REST calls, it's the most "raw".

The two step process you are demonstrating is another solution to the two step one I described. Presently, it appears there is no other way around it. Conceptually, if the templates are present on shared storage and visible from all nodes, then you could dispatch all the necessary commands from any single node.

So ultimately I'm asking, can you access templates from any node in the cluster (AFAIK, no) and if not, is that something that is being considered for future enhancements?

It appears from my vantage point that templates on shared storage are cluster wide resources and shouldn't be isolated to a single node (and the potential inaccessibility of them if that node is offline).

 

[bbgeek17]

In a 3 node cluster where VM100 is a template and is hosted on node1, I can:
- issue an API request directed to node2 to clone VM100 (which is located on node1) with target host node3. Provided I discover and use appropriate API path.
- mark VM100 as managed by HA and shutdown node1. The template will be moved to another host and will still be accessible

I think that answers your question of "can you access templates from any node in the cluster" as "yes".

---

Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[burly]

In a 3 node cluster where VM100 is a template and is hosted on node1, I can:
- issue an API request directed to node2 to clone VM100 (which is located on node1) with target host node3. Provided I discover and use appropriate API path.
- mark VM100 as managed by HA and shutdown node1. The template will be moved to another host and will still be accessible

I think that answers your question of "can you access templates from any node in the cluster" as "yes".

---

Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

I think I follow, thank you. I'll give it a whirl this weekend and update this post.

 

[burly]

I think I follow, thank you. I'll give it a whirl this weekend and update this post.

"This weekend" turned into two months due to a variety of delays on my end, but I'm happy to say this solution worked well. Thank you @bbgeek17 !

I used this post to make the templates HA: https://forum.proxmox.com/threads/managing-templates-ha.60417/