Combining custom cloud init with auto-generated

[PiotrD]

Hi,
Is it possible to somehow combine a custom cloud-init with an auto generated one ?

I thought that maybe if I do not set values that are autogenerated in my custom file, I will get them from ones I set in GUI. For example in my cicustom file for user-data I set some puppet and apt config

#cloud-config
apt:
sources:
puppet6:
keyid: "EF8D349F"
source: "deb http://apt.puppetlabs.com $RELEASE puppet6"
puppet:
install: true
version: "6.10.0-1bionic"
conf_file: "/etc/puppetlabs/puppet/puppet.conf"
ssl_dir: "/var/lib/puppet/ssl"
package_name: "puppet-agent"
conf:
agent:
server: "***-foreman01-**"
certname: "%f"
ca_cert: |
-------BEGIN CERTIFICATE-------
MIIFZTCCA02g..........

but I also want host name from GUI

hostname: my-hostname
manage_etc_hosts: true
fqdn: my-hostname-fqdn

Unfortunately, it doesn't combine these. When I set custom file, I do not get any values from auto generated part. Even if I don't set them in cicustom. I get default ones, e.g hostname is ubuntu. Not the name I set for this VM in a first place.

If it was possible, I would add this custom one config to a Template VM, and every time I cloned it, I would get puppet config from it and a hostname I specified during cloning.

 

[Alwin]

If it was possible, I would add this custom one config to a Template VM, and every time I cloned it, I would get puppet config from it and a hostname I specified during cloning.

You could use qemu-guest-agent to get these information from the VM.

 

[PiotrD]

You could use qemu-guest-agent to get these information from the VM.

Hmm, but I want to provide these data to the VM. My template VM is cloud-init ready image provided by Ubuntu/CentOS and I want set both hostname and additional user-data like puppet/apt configuration. Hostname, in auto-generated user-data, is set based on VM Name, and I would like to combine it somehow with other config, like mentioned puppet or apt config. Unfortunately, it is not combined or merged

But maybe I do not know some qemu-guest-agent feature that would allow to set this data on this VM ?

 

[Alwin]

But maybe I do not know some qemu-guest-agent feature that would allow to set this data on this VM ?

Check out man qm.

qm guest cmd <vmid> <command>
qm guest exec-status <vmid> <pid>
qm guest passwd <vmid> <username> [OPTIONS]
qm guest exec <vmid> [<extra-args>] [OPTIONS]

 

[PiotrD]

Check out man qm.

qm guest cmd <vmid> <command>
qm guest exec-status <vmid> <pid>
qm guest passwd <vmid> <username> [OPTIONS]
qm guest exec <vmid> [<extra-args>] [OPTIONS]

Hmm I know these commands, but it would only give me a console way to pass this data. Not via GUI, like with simple clone feature available in Proxmox :/ and the best way IMHO would be with a custom and an autogenerated config merge with cloud-init.

 

[Alwin]

You may certainly add a feature request for it. But if you have already a custom cloud-init, can't it be loaded from a different URL?
https://bugzilla.proxmox.com/

 

[dcsapak]

what you already can do is to add a 'hookscript' which creates and writes a new custom cloudinit config to the snippet used by cicustom
see
man qm

and especially the hookscript and 'qm cloudinit dump'

 

[PiotrD]

You may certainly add a feature request for it. But if you have already a custom cloud-init, can't it be loaded from a different URL?
https://bugzilla.proxmox.com/

I am not sure what do you mean to load from a different URL. I set vm to use a cicustom from a snippet file.

what you already can do is to add a 'hookscript' which creates and writes a new custom cloudinit config to the snippet used by cicustom
see
man qm

and especially the hookscript and 'qm cloudinit dump'

Wow, great thing, I did not know about hookscripts. I will try to add cicustom file snippet generation in pre-start phase. Hmm, but I guess I will have to store a cicustom snippet file for each VM. Thank you.

 

[PiotrD]

what you already can do is to add a 'hookscript' which creates and writes a new custom cloudinit config to the snippet used by cicustom
see
man qm

and especially the hookscript and 'qm cloudinit dump'

Hi, I have a problem with this kind of approach, I cannot edit already started VM.

My code looks like this

if ($phase eq 'pre-start') {

    # First phase 'pre-start' will be executed before the guest
    # ist started. Exiting with a code != 0 will abort the start

    print "$vmid is starting, doing preparations.\n";

    my $vm_user_data = `qm cloudinit dump $vmid user`;
    my $additional_user_data = `cat /var/lib/vz/snippets/user-data`;
    print $vm_user_data;
    print $additional_user_data;

    my $cloudinit_filename = "/var/lib/vz/snippets/cloudinit-$vmid";
    open my $fh, ">", $cloudinit_filename or die("Could not open file. $!");
    print $fh $vm_user_data;
    print $fh $additional_user_data;
    close $fh;
    my $log = `qm set $vmid --cicustom user=local:snippets/cloudinit-$vmid`;
    print $log;
    # print "preparations failed, aborting."
    # exit(1);

When I start this I cannot set clcustom, because this vm conf is locked

I get this error

trying to acquire lock...
can't lock file '/var/lock/qemu-server/lock-101.conf' - got timeout
GUEST HOOK: 101 post-start
101 started successfully.

Is it possible to edit vm conf in pre-start phase ? Or should you unlock it with qm unlock and lock it somehow ?

 

[dcsapak]

i would just set the snippets beforehand, and modifying their content....

 

[PiotrD]

i would just set the snippets beforehand, and modifying their content....

Hmm, thats one option, however it would not work properly with a template in some cases. If I attach hookscript and clcustom to a template and then clone it , all new VMs, based on it, would use the same cicustom file. So everytime I start a new VM, this shared file would be edited. Therefore, there is a small chance that two VMs started at the same time, would get the same config, am I right ?

 

[dcsapak]

yeah so you have to change the snippet file after cloning...

 

[PiotrD]

yeah so you have to change the snippet file after cloning...

Yeah, that's what I wanted to avoid. Manual step. I was hoping to just clone a VM in GUI, start and it is all ready. Is there maybe a some way to attach some script to clone procedure ? and attach empty cloutinit-$vmid file to newly cloned vm

 

[dcsapak]

ok one thing you could do (but please check that you know what you are doing) is to use the 'skiplock' parameter of qm set
with this, you skip the lock checks and should be able to set the config, but use it on your own risk...

 

[PiotrD]

ok one thing you could do (but please check that you know what you are doing) is to use the 'skiplock' parameter of qm set
with this, you skip the lock checks and should be able to set the config, but use it on your own risk...

Hmm, I tried this and it seems like it stills try to acquire lock:

system("qm set $vmid --skiplock  --cicustom user=local:snippets/cloudinit-$vmid");
sleep 5;

and I still get

trying to acquire lock...
can't lock file '/var/lock/qemu-server/lock-101.conf' - got timeout

 

[dcsapak]

Hmm, I tried this and it seems like it stills try to acquire lock:

ok the last resort is to manually modifying the config file... (i cannot recommend this for production use, so use at your own risk)

 

[PiotrD]

ok the last resort is to manually modifying the config file... (i cannot recommend this for production use, so use at your own risk)

Unfortunately, it won't work either.
I did sth like this in my hookscript

`sed -i '/cicustom:/c\\cicustom: user=local:snippets/cloudinit-$vmid' /etc/pve/qemu-server/$vmid.conf

It overwrites old cicustom file from a vm config (which is called cloudinit-empty). But it does it after VM is started. Even though, it is called pre-start phase VM has already loaded its config, so overwriting it afterwards won't help me. It still will use old cloud-init file.

I guess there is not any other option to reload this on this VM?

 

[dcsapak]

ok, then it seems that it just does not work this way at the moment, besides updating the cicustom file after cloning...

 

[PiotrD]

ok, then it seems that it just does not work this way at the moment, besides updating the cicustom file after cloning...

Ok.
That's a pity. Where can I send a feature request ?
FYi, I think the best would be to combine, both custom and generated one cloud-init file. Therefore, I could set some config in GUI and use custom part from a file.

 

[dcsapak]

you can open an enhancement request here https://bugzilla.proxmox.com/

FYi, I think the best would be to combine, both custom and generated one cloud-init file. Therefore, I could set some config in GUI and use custom part from a file.

yes i think that make sense if it is not too hard to implement...