Proxmox ignores custom cloud-init from --cicustom paramter (tested with Ubuntu/CentOS)

tina

New Member
Oct 26, 2020
5
0
1
40
Hi,

I'm getting crazy to create a vm provisioned with cloud-init. Here is what i do:


Bash:
# create template
wget https://cloud.centos.org/centos/8/x86_64/images/CentOS-8-GenericCloud-8.2.2004-20200611.2.x86_64.qcow2
qm create 9000 --name centos-ci  --memory 2048 --net0 virtio,bridge=vmbr0
qm importdisk 9000 CentOS-8-GenericCloud-8.2.2004-20200611.2.x86_64.qcow2 local-lvm
qm set 9000 --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-9000-disk-0
qm set 9000 --ide2 local-lvm:cloudinit
qm set 9000 --boot c --bootdisk scsi0
qm set 9000 --serial0 socket --vga serial0
qm template 9000

#create vm
qm clone 9000 300 --full true --name cli-test-cos --storage local-lvm
qm set 300 --cicustom user=local:snippets/user.yaml
qm start 300

The content of the /var/lib/zv/snippets/user.yaml
is

YAML:
chpasswd:
  expire: False
users:
  - k3s
    passwd: "$5$=4096$utiyPEPcWUUEiS7O33k2tmFP55zf2rK7KEYGuZn3zf6"
    sudo: "ALL=(ALL) NOPASSWD:ALL"
    ssh_authorized_keys:
      - "ssh-rsa AAAAB3NzaC1yc-....."

What i now expect is that the vm loads the user.yaml and setup the user/password. But nothing of this happens. I've also tried the same with ubuntu with this image.
In the "Cloud-Init" tab of the UI tells me, that no specific config is set:


1603745161363.png

while `qm config 300` looks like this:

Code:
boot: c
bootdisk: scsi0
cicustom: user=local:snippets/user.yaml
ide2: local-lvm:vm-300-cloudinit,media=cdrom,size=4M
memory: 2048
name: cli-test-cos
net0: virtio=F2:6D:2D:B3:87:C5,bridge=vmbr0
scsi0: local-lvm:vm-300-disk-0,size=10G
scsihw: virtio-scsi-pci
serial0: socket
smbios1: uuid=f7ab1f62-718c-4d70-9603-3c10858b7b6e
vga: serial0
vmgenid: a6520f50-d8ef-4ac4-8e68-ae80d48a0d6f

What addition steps i have to to to get the vm provisioned as expected?


Best regards
Tina
 
Last edited:
Try running cloud-init clean and then reboot.
 
Try running cloud-init clean and then reboot.
Since I'm not able to login to the vm I'm not sure how to do so. Of couse i could additionally set the -ciuser and -sshkeys opion but i thought the cicustom with the user.yaml should do the same...
 
Using a custom user snippet overrides the complete user config set in the GUI or config. This means you can't combine those 2 together. If you set ciuser, cipassword or sshkeys in the GUI and add a custom user snippet, only the options set in the snippet will be used.
 
Using a custom user snippet overrides the complete user config set in the GUI or config. This means you can't combine those 2 together. If you set ciuser, cipassword or sshkeys in the GUI and add a custom user snippet, only the options set in the snippet will be used.
Of course i know that I can't use both options at the same time...
But when i take a look into the cloud-init image i think i've seen my problem:

while the autogenerated config of proxmox by setting ciuser / cipassword looks like this:


YAML:
#cloud-config
hostname: cli-test-cos1
manage_etc_hosts: true
user: k3s
password: $5$i/sHvvmO$fsvuXzdWohMkFX0HZ13SO0/OwcRQhhpT0NhcobjAA74
chpasswd:
  expire: False
users:
  - default
package_upgrade: true

mine looks like this ( i followed the docs here)

YAML:
chpasswd:
  expire: False
users:
  - k3s
    passwd: "$5$=4096$utiyPEPcWUUEiS7O33k2tmFP55zf2rK7KEYGuZn3zf6"
    sudo: "ALL=(ALL) NOPASSWD:ALL"
    ssh_authorized_keys:
      - "ssh-rsa AAAAB3NzaC1yc-....."
 
@mira I've played around a little bit the the options:

I've set ciuser/cipasswd/sshkeys/nameserver/ipconfig0 options and all is working as expected. So i've take a look into the cloud-init iso and the configuration files:

meta-data
YAML:
instance-id: d57a4b93a50e48122186ddbe185efe1935a47741

network-config
YAML:
version: 1
config:
    - type: physical
      name: eth0
      mac_address: 'de:e8:f6:ef:13:fc'
      subnets:
      - type: static
        address: '192.168.20.150'
        netmask: '255.255.255.0'
        gateway: '192.168.20.1'
    - type: nameserver
      address:
      - '192.168.20.200'
      - '192.168.20.1'
      search:
      - 'domain.tld'

user-data
YAML:
hostname: cli-test-cos
manage_etc_hosts: true
fqdn: cli-test-cos.domain.tld
user: k3s
password: $5$98FxBt7m$h62.GEDIc4m1rcWt1IDjFsvVfh8g5WanTkGTdk1H0M3
ssh_authorized_keys:
  - ssh-rsa AAAA.....
chpasswd:
  expire: False
users:
  - default
package_upgrade: true

I've copied the code of the network-config and the user-data and put them under /var/lib/vz/snippets/(user|network).yaml
Then i've deleted the container and created it again:

Bash:
 qm clone 9000 300 --full true --name cli-test-cos --storage local-lvm
 qm set 300 -cicustom "user=local:snippets/user.yaml,network=local:snippets/network.yaml"
 qm start 300 # told me that the cloud-init iso is generated...


So the config matches exactly the one of the manual settings, but the config doesn't seems to be respected.
I've mounted the cloud-init iso on the pve host to check the content, but all configs are there and correct.
So it seems that the -set options do anything different to the vm OR the -cicustom option is broken.
 
I have the same problem - providing config files with
Code:
qm set 138 --cicustom "network=local:snippets/138-network.yml,user=local:snippets/138-user.yml"
and Debian gets stucked on:

Code:
[    4.122675] cloud-init[209]: SHA256:vnpWXRTWMVUM59cjPChKnfxp0g3YVj+pcrez0Fik7tk root@debian
[    4.124201] cloud-init[209]: The key's randomart image is:
[    4.125326] cloud-init[209]: +---[ECDSA 256]---+
[    4.126355] cloud-init[209]: |             .+BB|
[    4.127431] cloud-init[209]: |        o + +.oo*|[  OK  ] Started Initial cloud-init job (pre-networking).
[  OK  ] Reached target Network (Pre).


[  OK  ] Started ifup for eth0.
[    4.130813] cloud-init[209]: |       . * = = *+|
         Starting Raise network interfaces...


[   ***] A start job is running for Raise ne…rk interfaces (1min 33s / 5min 1s)

Why network settings is not set? I don't have DHCP so booting is stucked. Thanks.
 
Did you run cloud-init clean and then reboot?
 
I have the same problem - providing config files with
Code:
qm set 138 --cicustom "network=local:snippets/138-network.yml,user=local:snippets/138-user.yml"
and Debian gets stucked on:

Code:
[    4.122675] cloud-init[209]: SHA256:vnpWXRTWMVUM59cjPChKnfxp0g3YVj+pcrez0Fik7tk root@debian
[    4.124201] cloud-init[209]: The key's randomart image is:
[    4.125326] cloud-init[209]: +---[ECDSA 256]---+
[    4.126355] cloud-init[209]: |             .+BB|
[    4.127431] cloud-init[209]: |        o + +.oo*|[  OK  ] Started Initial cloud-init job (pre-networking).
[  OK  ] Reached target Network (Pre).


[  OK  ] Started ifup for eth0.
[    4.130813] cloud-init[209]: |       . * = = *+|
         Starting Raise network interfaces...


[   ***] A start job is running for Raise ne…rk interfaces (1min 33s / 5min 1s)

Why network settings is not set? I don't have DHCP so booting is stucked. Thanks.
This is a common problem if you set IPv6 to DHCP without having a DHCP server.
 
@tina

I know it's little old thread but I was having similar problem. It would not pick up my custom snippet. What did the trick in my case is the addition of "#cloud-config" as the first line of the yml file:

A sample that is working on my end is as following:
YAML:
#cloud-config
hostname: pkshr
manage_etc_hosts: true
user: ubuntu
chpasswd:
  expire: False
users:
  - default
ssh_authorized_keys:
  - ssh-rsa XXXXXXXXXXXXXXXXXXXXXx
package_upgrade: true
 
@tina

I know it's little old thread but I was having similar problem. It would not pick up my custom snippet. What did the trick in my case is the addition of "#cloud-config" as the first line of the yml file:

A sample that is working on my end is as following:
YAML:
#cloud-config
hostname: pkshr
manage_etc_hosts: true
user: ubuntu
chpasswd:
  expire: False
users:
  - default
ssh_authorized_keys:
  - ssh-rsa XXXXXXXXXXXXXXXXXXXXXx
package_upgrade: true
This worked ?
 
Ok it worked for me also but dump still shows stupid things that aren't related with configs.

Code:
root@pve:/var/lib/vz/snippets# qm cloudinit dump 100 user
#cloud-config
hostname: srv-app-1
manage_etc_hosts: true
chpasswd:
  expire: False
users:
  - default
package_upgrade: true

root@pve:/var/lib/vz/snippets# qm cloudinit dump 100 network
version: 1
config:
    - type: nameserver
      address:
      - '10.0.1.1'
      - '1.1.1.1'
      search:
      - 'hw.sonic'

For example my network-config looks like this

Code:
#cloud-config
version: 1
config:
    - type: physical
      name: ens18
      subnets:
        - type: static
          address: 10.0.101.1/24
          gateway: 10.0.101.254
    - type: nameserver
      address: ["10.0.1.30","1.1.1.1"]
      search: ["pve.sonic","hw.sonic","sonic"]
 
Last edited:
Ok it worked for me also but dump still shows stupid things that aren't related with configs.
the "qm cloudinit dump" only shows the PVE native cloudinit config, it does not look at or show custom files. If you are using custom cloudinit, the output of dump is completely irrelevant.


Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
  • Like
Reactions: mira

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!