Hi,
currently I have a vSphere with a VM template (changes only every few weeks), instantiated by a Jenkins build automation server (CI) for a job running few minutes up to one day and then is shutdown and discarded (there are no more than 50 instances at the same time and they store all needed data outside (i.e. Jenkins, own NAS mount...)).
I'm playing with a Proxmox cluster and essentially like to start the same. I didn't get the Jenkins Proxmox Plugin working yet, but I could use shell commands for now.
On a single node, this is very simple: just have a template and clone it as often as needed, DHCP and a boot script (running an agent) do the rest, straightforward and great, a pleasure to work with!
Now I'd like to start a few clones on each node. I noticed I can easily replicate the VM template using the replication feature, but then I don't know how to clone it on the target node: I see the disk being available in ZFS storage, but have no VM template there. I read that I could of course use shared storage, but I just have a 1Gbit attached NAS (more than sufficient for the small job results, but surely not when sharing virtual disk I/O among 10 or 20 nodes). I think I had to use a linked clone to avoid copying the whole thing from NAS, but still then each first read access needs to travel the slow network. But due to replication, the template should be available on the nodes (and most of its data - the big disk image - in fact already is), so I think it should be possible to locally clone it fast - (almost) no network traffic needed.
NB: because this was suggested in an article with a similar question: Yes, I read about the HA feature and I think its great for example for the jenkins container: one instance is kept running, great! But I don't see how this could help me with the "cloned disposable workload" VMs, because let's say I have 50 on 10 nodes and they are "ephemeral".
I think I can write a script to migrate-copy the template to each node (every few weeks) and then (for each VM instance) clone from the appropriate copied template, but this feels wrong: I assume some mechanism exists for that. Also there is support for cloudinit, which is often used for such scenarios.
What is the best way to run a number of VMs from one and the same template, distributed over multiple nodes?
Any hint or pointer appreciated!
Steffen
currently I have a vSphere with a VM template (changes only every few weeks), instantiated by a Jenkins build automation server (CI) for a job running few minutes up to one day and then is shutdown and discarded (there are no more than 50 instances at the same time and they store all needed data outside (i.e. Jenkins, own NAS mount...)).
I'm playing with a Proxmox cluster and essentially like to start the same. I didn't get the Jenkins Proxmox Plugin working yet, but I could use shell commands for now.
On a single node, this is very simple: just have a template and clone it as often as needed, DHCP and a boot script (running an agent) do the rest, straightforward and great, a pleasure to work with!
Now I'd like to start a few clones on each node. I noticed I can easily replicate the VM template using the replication feature, but then I don't know how to clone it on the target node: I see the disk being available in ZFS storage, but have no VM template there. I read that I could of course use shared storage, but I just have a 1Gbit attached NAS (more than sufficient for the small job results, but surely not when sharing virtual disk I/O among 10 or 20 nodes). I think I had to use a linked clone to avoid copying the whole thing from NAS, but still then each first read access needs to travel the slow network. But due to replication, the template should be available on the nodes (and most of its data - the big disk image - in fact already is), so I think it should be possible to locally clone it fast - (almost) no network traffic needed.
NB: because this was suggested in an article with a similar question: Yes, I read about the HA feature and I think its great for example for the jenkins container: one instance is kept running, great! But I don't see how this could help me with the "cloned disposable workload" VMs, because let's say I have 50 on 10 nodes and they are "ephemeral".
I think I can write a script to migrate-copy the template to each node (every few weeks) and then (for each VM instance) clone from the appropriate copied template, but this feels wrong: I assume some mechanism exists for that. Also there is support for cloudinit, which is often used for such scenarios.
What is the best way to run a number of VMs from one and the same template, distributed over multiple nodes?
Any hint or pointer appreciated!
Steffen
Last edited: