[SOLVED] cloud-init custom network configuration does not get applied

J

jschwab

New Member
Jan 23, 2023
5
0
1
I've been using the --cicustom feature for some time already without issues, but recently it doesn't work any more.

File at pvesm path ceph_fs:snippets/network-myvm.yaml

YAML: 
network:
  version: 2
  ethernets:
    eth_a:
      set-name: eth_a
      match:
        macaddress: BC:24:11:08:32:3D
      dhcp4: true
    eth_b:
      set-name: eth_b
      match:
        macaddress: 02:00:0a:7f:ef:dc
      addresses:
      - 10.127.239.220/24
      nameservers:
        addresses:
        - 10.127.239.251
        - 10.127.239.252
        - 10.127.239.253
        search:
        - mydomain.local

I create a new VM from a Debian 12 image, setting the yaml file as custom network configuration:

Code: 
qm create $vm_id --memory 2048 --scsihw virtio-scsi-pci --cores 2 --cpu EPYC"
qm set    $vm_id --scsi0 ceph_rbd:0,import-from=/mnt/pve/ceph_fs/template/iso/debian-12-genericcloud-amd64.qcow2,discard=on
qm set    $vm_id --agent enabled=1,freeze-fs-on-backup=1,fstrim_cloned_disks=1
qm set    $vm_id --name '${vm_name}'
qm set    $vm_id --ide2 ceph_rbd:cloudinit
qm set    $vm_id --boot order=scsi0
qm set    $vm_id --serial0 socket --vga serial0
qm set    $vm_id --sshkeys /root/id_ed25519.pub
qm set    $vm_id --ciuser root
qm set    $vm_id --net0 model=virtio,macaddr=02:00:0a:7f:ef:dc,bridge=vmbr0,tag=4005
qm set    $vm_id --net1 model=virtio,macaddr=BC:24:11:08:32:3D,bridge=vmbr0,tag=2455
qm set    $vm_id --cicustom 'network=ceph_fs:snippets/network-myvm.yaml

During booting, I see this error:

Code: 
[    3.117363] cloud-init[360]: 2024-02-22 13:56:09,952 - networking.py[WARNING]: Not all expected physical devices present: {'BC:24:11:08:32:3D'}
[    3.119054] cloud-init[360]: 2024-02-22 13:56:09,952 - util.py[WARNING]: failed stage init-local
[    3.132126] cloud-init[360]: failed run of stage init-local
[    3.132975] cloud-init[360]: ------------------------------------------------------------
[    3.134067] cloud-init[360]: Traceback (most recent call last):
[    3.134849] cloud-init[360]:   File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 767, in status_wrapper
[    3.136212] cloud-init[360]:     ret = functor(name, args)
[    3.136938] cloud-init[360]:           ^^^^^^^^^^^^^^^^^^^
[    3.137666] cloud-init[360]:   File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 433, in main_init
[    3.138974] cloud-init[360]:     init.apply_network_config(bring_up=bring_up_interfaces)
[    3.140055] cloud-init[360]:   File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 924, in apply_network_config
[    3.141452] cloud-init[360]:     self.distro.networking.wait_for_physdevs(netcfg)
[    3.142412] cloud-init[360]:   File "/usr/lib/python3/dist-packages/cloudinit/distros/networking.py", line 179, in wait_for_physdevs
[    3.143904] cloud-init[360]:     raise RuntimeError(msg)
[    3.144620] cloud-init[360]: RuntimeError: Not all expected physical devices present: {'BC:24:11:08:32:3D'}
[    3.145847] cloud-init[360]: ------------------------------------------------------------

So the custom network configuration seems to get parsed, but not applied because it can't find the device or something like that.
When I login to the VM via console, I see that /etc/netplan is empty and both network devices are present but uncofigured. If I now manually place the yaml from above into /etc/netplan, everything works as it should.

Has someone an idea what might be the issue?
 
You have to troubleshoot your issue from the VM perspective. The only involvement PVE, as hypervisor, has is to provide network interfaces with specified MACs and ISO with yaml file. Its possible you have a race in the VM, and the network devices dont get initialized in time. It could be a kernel or configuration change of your VM. It could be the way you built your image (if you are not using standard cloud image).

You need to examine the boot log, note the appearance/renaming/modifications of network devices. Examine cloud-init log in detail. Try to apply/re-apply sections of cloud-init on booted/rebooted system.

It could be a bug in a new package that got pulled, it could be a bug in old package - https://bugs.launchpad.net/cloud-init/+bug/1846535 . Again, not in PVE control.

Its very unlikely that your experience can be improved with a change to PVE configuration in any way.

Good luck.

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
Thank you very much for the quick reply! Funnily enough, making the mac addresses lower case and quoting them seems to have made the trick. I've now spun up several new VMs without a hitch.
 
It's just kind of unexpected that such a thing could break cloud-init, since writing macs in lower case is not uncommon and it's pretty trivial to parse them case-insensitive.
 
You must log in or register to reply here.
Top Bottom