Cloud-init sets network config; --cicustom is cleared; network config erased on next boot

[sebastian19276]

I have cephfs:snippets set up with cloud-init files. This is for creating Ubuntu 20.04 VMs. Here's an example:

# cat /mnt/pve/cephfs/snippets/net107.yaml
version: 1
config:
    - type: physical
      name: eth0
      mac_address: '36:9d:01:aa:42:d8'
      subnets:
      - type: static
        address: '192.168.122.107'
        netmask: '255.255.255.0'
        gateway: '192.168.122.1'
    - type: nameserver
      address:
      - '192.168.122.104'
      search:
      - 'example.com'

Then I use it like this:

# qm create 107 --net0 virtio,bridge=vmbr0 --sockets 1 --cores 2 --memory 2048 --name testvm
# qm set 107 --cicustom "network=cephfs:snippets/net107.yaml"
# qm start 107
# qm set 107 --cicustom ""

This works the first time. The VM boots and it gets a network config. However it doesn't persist across reboots... I have to add this back. Why is this? I thought cloud-init runs on first boot and afterwards it's good to go. Why would I have to maintain the existence of this network config file - why wouldn't this be ingested into the VM and stay there? I have the guest agent installed and it reports no network configuration...

Edit: forgot --cicustom in the second line of the qm commands.

 

[sebastian19276]

It appears I was caught by this bug
https://bugzilla.redhat.com/show_bug.cgi?id=1593010
There isn't a great fix in there except for this comment https://bugzilla.redhat.com/show_bug.cgi?id=1593010#c39

fwiw, adding the following as `custom script` in `Use Cloud-Init` RHV UI works, the VM will get the correct static IP information and will build the ifcfg-ethX correctly, then will disable network configuration from cloud-init

~~~
#cloud-config

runcmd:
 - [ sh, -c, 'echo "network: {config: disabled}" >> /etc/cloud/cloud.cfg' ]
~~~

This did fix my issue. My network config now persists after I remove the cloud-init drive.

 

[mira]

If you remove the network config afterwards, the instance id changes (since the config changes) and it will reconfigure the network.
You'll have to keep the network config part.

 

[sebastian19276]

If you remove the network config afterwards, the instance id changes (since the config changes) and it will reconfigure the network.
You'll have to keep the network config part.

Hi, thank you for the reply. That part was made clear in the red hat bug report. The issue is the network config is backed by an actual file on disk. I would like to not have a config file on disk remain behind after creating a VM and be a dependency for that VM coming back online after a reboot.