Combining custom cloud init with auto-generated
- Thread starter PiotrD
- Start date
Wouldn't it be sufficient to introduce some method to pass the VMID to the custom cloud-init file?
Then one could have a (dynamically generated) cicustom file for every VM and the template can be cloned without any modifications afterwards.
*or*
A hook for a new vm being created from template is added so we have additional states to "pre-start",...
Something like "post-clone" (when the config isn't locked anymore) would be great.
Then one could have a (dynamically generated) cicustom file for every VM and the template can be cloned without any modifications afterwards.
*or*
A hook for a new vm being created from template is added so we have additional states to "pre-start",...
Something like "post-clone" (when the config isn't locked anymore) would be great.
Hi,
sorry to tag on to this,
but was the idea of using a custom cloud-init script and the generated one together possible?
i have a simple cloud-init file which just simply adds the package qemu-guest-agent and then reboots after,
but every time i add it in like so
it just seems to IGNORE the auto generated cloud-init from the proxmox gui and only run my script?
sorry to tag on to this,
but was the idea of using a custom cloud-init script and the generated one together possible?
i have a simple cloud-init file which just simply adds the package qemu-guest-agent and then reboots after,
but every time i add it in like so
Code:
qm set 102 --cicustom "user=local:snippets/userconfig.yaml"it just seems to IGNORE the auto generated cloud-init from the proxmox gui and only run my script?
Hey there, I was also attempting a similar project and ran across this thread.
I have created the template for the cloud-init but I have the opposite issue, it ignores my custom snippet.
@si458, would you mind sharing some details about how you have configured your setup for custom cloudinit? I think I might be doing something wrong but I dont know what.
I have created the template for the cloud-init but I have the opposite issue, it ignores my custom snippet.
@si458, would you mind sharing some details about how you have configured your setup for custom cloudinit? I think I might be doing something wrong but I dont know what.
Hi
I’m not doing anything special
I followed the cloud init guide from pve.proxmox.com using the latest Ubuntu 20.04 cloud kvm image but ignored the custom script step
when i cloned the template I set details etc in the cloud init gui tab then clicked generate image and watched the console tab and saw it do it’s updates set hostname ip etc
So then i deleted the cloned vm and ran the custom script line to add it to the template and then recloned, set details etc then ran n watched the console and it just totally ignored all the automated gui stuff and only ran what I set in my custom config which was to add a single package, it totally ignored the set hostname, user, etc
I did however yesterday have a look at the source code for the cloudinit stuff in proxmox and it seems when I click generate image it checks if set a custom script and if so use that for the iso rather than using the autogenerated one from the gui and the custom one
I’m not doing anything special
I followed the cloud init guide from pve.proxmox.com using the latest Ubuntu 20.04 cloud kvm image but ignored the custom script step
when i cloned the template I set details etc in the cloud init gui tab then clicked generate image and watched the console tab and saw it do it’s updates set hostname ip etc
So then i deleted the cloned vm and ran the custom script line to add it to the template and then recloned, set details etc then ran n watched the console and it just totally ignored all the automated gui stuff and only ran what I set in my custom config which was to add a single package, it totally ignored the set hostname, user, etc
I did however yesterday have a look at the source code for the cloudinit stuff in proxmox and it seems when I click generate image it checks if set a custom script and if so use that for the iso rather than using the autogenerated one from the gui and the custom one
Hmmm.....thats interesting.
After you add the custom init to the template, if you do a qm cloudinit dump <vm id> user does it show you your custom file?
I just tried building a new template with the ubuntu 20.04 image and tried to add a custom init but it looks like the system wont even take it. I am nto sure if it is because I am doing a step out of order or if it is because proxmox doesnt like my init file.
After you add the custom init to the template, if you do a qm cloudinit dump <vm id> user does it show you your custom file?
I just tried building a new template with the ubuntu 20.04 image and tried to add a custom init but it looks like the system wont even take it. I am nto sure if it is because I am doing a step out of order or if it is because proxmox doesnt like my init file.
Hi,
sadly no it doesnt it only shows the generated cloud-init from the GUI, but again if you set a custom script it totally ignores it
HOWEVER a previous post shows a way around it for the moment, im writing my own at min
but it basically reads each config file for each VM then generates a YAML which dumps the generated cloud-init from proxmox and then includes your custom config, and you then instruct proxmox to use the generated yaml instead of your custom one
which isnt bad but alot of hassle if you have lots of VMs
sadly no it doesnt it only shows the generated cloud-init from the GUI, but again if you set a custom script it totally ignores it
HOWEVER a previous post shows a way around it for the moment, im writing my own at min
but it basically reads each config file for each VM then generates a YAML which dumps the generated cloud-init from proxmox and then includes your custom config, and you then instruct proxmox to use the generated yaml instead of your custom one
which isnt bad but alot of hassle if you have lots of VMs
Hello,
if you know golang, i have a made a very basic and simple script that simply does custom cloudinit files as a work around
you must however run it every time you change the config either in proxmox or your custom script
https://github.com/si458/proxmox-customcloudinit/
if you know golang, i have a made a very basic and simple script that simply does custom cloudinit files as a work around
you must however run it every time you change the config either in proxmox or your custom script
https://github.com/si458/proxmox-customcloudinit/
Why pve doesn't implement a full-functional cloud-init web config support? It's so easy.
Such as windows DNS support, cloudbase-init windows password support, apt/yum source support.
This is useful everywhere.
If it's not written in perl, in a modern program language, I can write it in a week.
Such as windows DNS support, cloudbase-init windows password support, apt/yum source support.
This is useful everywhere.
If it's not written in perl, in a modern program language, I can write it in a week.
Hello,
We're running on the same limitation.
Just a thought : couldn't it be easier to implement the vendor-data file inside the cloud-init image ?
It would probably only need to add a few lines of code inside QemuServer/Cloudinit.pm, functions generate_nocloud() and get_custom_cloudinit_files(). At least for the 'nocloud' disk. Perhaps a few others for the configdrive2 too.
commit_cloudinit_disk() is not checking file's list so nothing should be changed in this one.
I'll try to set up a dev environnement to check the idea and comment here.
We're running on the same limitation.
Just a thought : couldn't it be easier to implement the vendor-data file inside the cloud-init image ?
It would probably only need to add a few lines of code inside QemuServer/Cloudinit.pm, functions generate_nocloud() and get_custom_cloudinit_files(). At least for the 'nocloud' disk. Perhaps a few others for the configdrive2 too.
commit_cloudinit_disk() is not checking file's list so nothing should be changed in this one.
I'll try to set up a dev environnement to check the idea and comment here.
Hello,
Here's a small patch that add support for vendor-data config file which is actually not used by Proxmox in the cloud-init generated defaults. So you can keep generated network-data, user-data and meta-data and add your own config.
Note : this patch applies against qemu-server version 6.1-7. Not exactly the last version.
Beware : per cloud-init doc, vendor-data is ALWAYS superseded by user-data (https://cloudinit.readthedocs.io/en/latest/topics/vendordata.html)
Warning : it has not been extensively tested : all Proxmox daemons restart fine after applying the patch but live-migration and those 'advanced features' has been tested.
As always, volunteers welcome.
After applying the patch to the /usr/share/perl5/PVE/ tree, you simply have to specify a custom cloud-init config like this :
I'll now have to read the dev documentation to understand how to submit this patch for a future release. If a Promox dev is kind enough to check the patch, apply it and test it, I would owe a cofee ;-)
Here's a small patch that add support for vendor-data config file which is actually not used by Proxmox in the cloud-init generated defaults. So you can keep generated network-data, user-data and meta-data and add your own config.
Note : this patch applies against qemu-server version 6.1-7. Not exactly the last version.
Beware : per cloud-init doc, vendor-data is ALWAYS superseded by user-data (https://cloudinit.readthedocs.io/en/latest/topics/vendordata.html)
Warning : it has not been extensively tested : all Proxmox daemons restart fine after applying the patch but live-migration and those 'advanced features' has been tested.
As always, volunteers welcome.
After applying the patch to the /usr/share/perl5/PVE/ tree, you simply have to specify a custom cloud-init config like this :
Code:
cicustom: vendor=local:snippets/vendor.yamlI'll now have to read the dev documentation to understand how to submit this patch for a future release. If a Promox dev is kind enough to check the patch, apply it and test it, I would owe a cofee ;-)
That is a fantastic patch! I ended up at this thread with the same challenge. I wanted a custom user data but wanted to set the hostname via the clone process. This simple patch allows me to add all the functionality common to all my new systems (e.g. qemu-guest-agent install) but not have to manually edit the file for every clone.
I applied the patch to version 6.3-3 without any issue. I also updated the /usr/share/perl5/PVE/API2/Qemu.pm file so that the dump command would not fail on vendor. Doesn't really do anything but won't fail. New patch attached
I have tested this with a small 3 node cluster using an external Ceph setup for the shared disk storage and CephFS for the shared snippets.
All the normal functions work without issue (start, stop, restart, all statistics) in both standalone and HA configuration. Live Migration works without issue. I haven't tried HA failover yet because I don't feel like shutting down one of my nodes at the moment but have zero reason, based on the other testing that there will be any issues.
I applied the patch to version 6.3-3 without any issue. I also updated the /usr/share/perl5/PVE/API2/Qemu.pm file so that the dump command would not fail on vendor. Doesn't really do anything but won't fail. New patch attached
I have tested this with a small 3 node cluster using an external Ceph setup for the shared disk storage and CephFS for the shared snippets.
All the normal functions work without issue (start, stop, restart, all statistics) in both standalone and HA configuration. Live Migration works without issue. I haven't tried HA failover yet because I don't feel like shutting down one of my nodes at the moment but have zero reason, based on the other testing that there will be any issues.
Please be aware that manual changes will get overwritten on package update.
Please post the patch to pve-devel for further integration, if wished.
https://pve.proxmox.com/wiki/Developer_Documentation
Please post the patch to pve-devel for further integration, if wished.
https://pve.proxmox.com/wiki/Developer_Documentation
After I send the patch to devel @mira told me she already send it on june 24 and it's waiting for testing.
I added an example vendor config to https://bugzilla.proxmox.com/show_bug.cgi?id=2429#c7
I added an example vendor config to https://bugzilla.proxmox.com/show_bug.cgi?id=2429#c7
Last edited: