[SOLVED] cloud-init + ubuntu + template + vm + uefi + customization

M

morik_proxmox

New Member
Jan 2, 2023
13
0
1
Hello everyone,
I'm a recent vmware-->proxmox convert. So, please bear with my ignorance.
Goal: Create cloud-init vm template customized (at parity) with my vmware cloud-init scripts to make cloning/creating new VMs an easy operation.

Issue: I'm able to get an ubuntu 22.04 clout-init template with bios: seabios to work fine. But, when creating uefi template, the cloned vm hangs up, until issuance of shutdown command, after which cloudinit resumes installation.

system: pve-manager/8.2.4/faa83925c9641325 (running kernel: 6.8.8-3-pve), cluster, no CEPH, local + local-lvm on SSDs running on Intel i11/i14 NUCs

cloud image: jammy-server-cloudimg-amd64.img with following modifications
Code: 
qemu-img resize /var/lib/vz/template/iso/jammy-server-cloudimg-amd64.img 32G
virt-customize -a /var/lib/vz/template/iso/jammy-server-cloudimg-amd64.img --install qemu-guest-agent

uefi vm template creation commands:
Code: 
#!/bin/bash
qm create 9001 --name "t-vm-ubu-22.04-uefi-v01" --ostype l26 \
    --memory 2048 \
    --agent 1 \
    --bios ovmf --efidisk0 local-lvm:0,pre-enrolled-keys=0 \
    --socket 1  \
    --vga serial0 --serial0 socket  \
    --net0 virtio,bridge=vmbr0,tag=100
qm importdisk 9001 /var/lib/vz/template/iso/jammy-server-cloudimg-amd64.img local-lvm
qm set 9001 --scsihw virtio-scsi-pci --virtio0 local-lvm:vm-9001-disk-1,discard=on,size=35020M
qm set 9001 --ide2 local-lvm:cloudinit
qm set 9001 --boot order=virtio0
qm set 9001 --cicustom "vendor=local:snippets/esco.yaml,user=local:snippets/users.yaml,network=local:snippets/metadata_ubu-gold.yaml"
qm set 9001 --tags template,ubuntu-template,22.04,cloudinit,esco,ubuntu
qm set 9001 --ipconfig0 ip=dhcp
qm template 9001

(resulting) uefi vm template config:
Code: 
agent: 1
bios: ovmf
boot: order=virtio0
cicustom: vendor=local:snippets/esco.yaml,user=local:snippets/users.yaml,network=local:snippets/metadata_ubu-gold.yaml
efidisk0: local-lvm:base-9001-disk-0,pre-enrolled-keys=0,size=4M
ide2: local-lvm:vm-9001-cloudinit,media=cdrom
ipconfig0: ip=dhcp
memory: 2048
meta: creation-qemu=9.0.0,ctime=1721965830
name: t-vm-ubu-22.04-uefi-v01
net0: virtio=BC:24:11:40:2F:39,bridge=vmbr0,tag=100
ostype: l26
scsihw: virtio-scsi-pci
serial0: socket
smbios1: uuid=2428f3e8-cb09-4aaf-921b-521752594f24
sockets: 1
tags: 22.04;cloudinit;esco;template;ubuntu;ubuntu-template
template: 1
vga: serial0
virtio0: local-lvm:base-9001-disk-1,discard=on,size=32G
vmgenid: 5537f101-60ea-42e3-8df3-330af4158c86

linked cloned vm config:
Code: 
agent: 1
bios: ovmf
boot: order=virtio0
cicustom: vendor=local:snippets/esco.yaml,user=local:snippets/users.yaml,network=local:snippets/metadata_ubu-gold.yaml
efidisk0: local-lvm:vm-101-disk-0,pre-enrolled-keys=0,size=4M
ide2: local-lvm:vm-101-cloudinit,media=cdrom,size=4M
ipconfig0: ip=dhcp
memory: 2048
meta: creation-qemu=9.0.0,ctime=1721965830
name: ubu-gold
net0: virtio=BC:24:11:5D:E4:95,bridge=vmbr0,tag=100
ostype: l26
scsihw: virtio-scsi-pci
serial0: socket
smbios1: uuid=c798b700-b092-45ff-a838-dd88a4662786
sockets: 1
tags: 22.04;cloudinit;esco;template;ubuntu;ubuntu-template
vga: std
virtio0: local-lvm:vm-101-disk-1,discard=on,size=32G
vmgenid: 914e41c6-262b-40c9-809b-d92c48908128

Upon booting, vm hangs after Time and Date Service configuration for 2+ minutes. If i don't intervene, then default cloudinit prompt with default username password shows but no username/pwd combination is accepted. If I shutdown after that, system reboots and proceeds towards normal cloud-init installation.

vm syslog:
Code: 
Jul 26 03:51:50 ubuntu systemd[1]: Started snap.lxd.hook.configure-7d66ed4f-5355-4534-b360-277474c615ba.scope.
Jul 26 03:51:50 ubuntu systemd[1]: snap.lxd.hook.configure-7d66ed4f-5355-4534-b360-277474c615ba.scope: Deactivated successfully.
Jul 26 03:51:51 ubuntu dbus-daemon[476]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service' requested by ':1.10' (uid=0 pid=488 comm="/usr/lib/snapd/snapd " label="unconfined")
Jul 26 03:51:51 ubuntu systemd[1]: Starting Time & Date Service...
Jul 26 03:51:51 ubuntu dbus-daemon[476]: [system] Successfully activated service 'org.freedesktop.timedate1'
Jul 26 03:51:51 ubuntu systemd[1]: Started Time & Date Service.
Jul 26 03:51:53 ubuntu systemd[1]: dmesg.service: Deactivated successfully.
Jul 26 03:52:14 ubuntu systemd[1]: systemd-fsckd.service: Deactivated successfully.
Jul 26 03:52:21 ubuntu systemd[1]: systemd-timedated.service: Deactivated successfully.
Jul 26 03:52:23 ubuntu snapd[488]: stateengine.go:149: state ensure error: persistent network error: Get "https://api.snapcraft.io/api/v1/snaps/sections": dial tcp: lookup api.snapcraft.io: Temporary failure in name resolution
Jul 26 03:52:23 ubuntu dbus-daemon[476]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service' requested by ':1.10' (uid=0 pid=488 comm="/usr/lib/snapd/snapd " label="unconfined")
Jul 26 03:52:23 ubuntu systemd[1]: Starting Time & Date Service...
Jul 26 03:52:23 ubuntu dbus-daemon[476]: [system] Successfully activated service 'org.freedesktop.timedate1'
Jul 26 03:52:23 ubuntu systemd[1]: Started Time & Date Service.
Jul 26 03:52:23 ubuntu systemd[1]: Finished Wait until snapd is fully seeded.
Jul 26 03:52:23 ubuntu systemd[1]: Condition check resulted in Auto import assertions from block devices being skipped.
Jul 26 03:52:53 ubuntu dbus-daemon[476]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service' requested by ':1.10' (uid=0 pid=488 comm="/usr/lib/snapd/snapd " label="unconfined")
Jul 26 03:52:53 ubuntu systemd[1]: systemd-timedated.service: Deactivated successfully.
Jul 26 03:52:53 ubuntu systemd[1]: Starting Time & Date Service...
Jul 26 03:52:53 ubuntu dbus-daemon[476]: [system] Successfully activated service 'org.freedesktop.timedate1'
Jul 26 03:52:53 ubuntu systemd[1]: Started Time & Date Service.
Jul 26 03:53:23 ubuntu systemd[1]: systemd-timedated.service: Deactivated successfully.
Jul 26 03:53:23 ubuntu dbus-daemon[476]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service' requested by ':1.10' (uid=0 pid=488 comm="/usr/lib/snapd/snapd " label="unconfined")
Jul 26 03:53:23 ubuntu systemd[1]: Starting Time & Date Service...
Jul 26 03:53:23 ubuntu dbus-daemon[476]: [system] Successfully activated service 'org.freedesktop.timedate1'
Jul 26 03:53:23 ubuntu systemd[1]: Started Time & Date Service.
Jul 26 03:53:26 ubuntu qemu-ga: info: guest-shutdown called, mode: (null)
Jul 26 03:53:26 ubuntu systemd[1]: Removed slice Slice /system/modprobe.
Jul 26 03:53:26 ubuntu systemd[1]: Stopped target Host and Network Name Lookups.
Jul 26 03:53:26 ubuntu systemd[1]: apt-daily-upgrade.timer: Deactivated successfully.
Jul 26 03:53:26 ubuntu systemd[1]: Stopped target Host and Network Name Lookups.
Jul 26 03:53:26 ubuntu systemd[1]: apt-daily-upgrade.timer: Deactivated successfully.
Jul 26 03:53:26 ubuntu systemd[1]: Stopped Daily apt upgrade and clean activities.
Jul 26 03:53:26 ubuntu systemd[1]: apt-daily.timer: Deactivated successfully.
Jul 26 03:53:26 ubuntu systemd[1]: Stopped Daily apt download activities.
Jul 26 03:53:26 ubuntu systemd[1]: dpkg-db-backup.timer: Deactivated successfully.
Jul 26 03:53:26 ubuntu systemd[1]: Stopped Daily dpkg database backup timer.
Jul 26 03:53:26 ubuntu systemd[1]: e2scrub_all.timer: Deactivated successfully.
Jul 26 03:53:26 ubuntu systemd[1]: Stopped Periodic ext4 Online Metadata Check for All Filesystems.
Jul 26 03:53:26 ubuntu systemd[1]: fstrim.timer: Deactivated successfully.
Jul 26 03:53:26 ubuntu systemd[1]: Stopped Discard unused blocks once a week.
Jul 26 03:53:26 ubuntu systemd[1]: logrotate.timer: Deactivated successfully.
Jul 26 03:53:26 ubuntu systemd[1]: Stopped Daily rotation of log files.
Jul 26 03:53:26 ubuntu systemd[1]: man-db.timer: Deactivated successfully.
Jul 26 03:53:26 ubuntu systemd[1]: Stopped Daily man-db regeneration.
Jul 26 03:53:26 ubuntu systemd[1]: motd-news.timer: Deactivated successfully.
Jul 26 03:53:26 ubuntu systemd[1]: Stopped Message of the Day.
Jul 26 03:53:26 ubuntu systemd[1]: systemd-tmpfiles-clean.timer: Deactivated successfully.
Jul 26 03:53:26 ubuntu systemd[1]: Stopped Daily Cleanup of Temporary Directories.
Jul 26 03:53:26 ubuntu systemd[1]: lvm2-lvmpolld.socket: Deactivated successfully.
Jul 26 03:53:26 ubuntu systemd[1]: Closed LVM2 poll daemon socket.
Jul 26 03:53:26 ubuntu systemd[1]: systemd-rfkill.socket: Deactivated successfully.
Jul 26 03:53:26 ubuntu systemd[1]: Closed Load/Save RF Kill Switch Status /dev/rfkill Watch.
Jul 26 03:53:26 ubuntu systemd[1]: Stopping Create final runtime dir for shutdown pivot root...
Jul 26 03:53:26 ubuntu systemd[1]: Stopping QEMU Guest Agent...
Jul 26 03:53:26 ubuntu systemd[1]: Condition check resulted in Ubuntu core (all-snaps) system shutdown helper setup service being skipped.
Jul 26 03:53:26 ubuntu systemd[1]: Stopping Load/Save Random Seed...
Jul 26 03:59:43 ubu-gold systemd-random-seed[347]: Kernel entropy pool is not initialized yet, waiting until it is.
Jul 26 03:59:43 ubu-gold systemd[1]: Starting Flush Journal to Persistent Storage...
Jul 26 03:59:43 ubu-gold systemd[1]: Finished Flush Journal to Persistent Storage.
Jul 26 03:59:43 ubu-gold systemd-fsck[341]: cloudimg-rootfs: clean, 73905/274176 files, 459513/548091 blocks
Jul 26 03:59:43 ubu-gold multipathd[356]: --------start up--------
Jul 26 03:59:43 ubu-gold multipathd[356]: read /etc/multipath.conf
Jul 26 03:59:43 ubu-gold multipathd[356]: path checkers start up
Jul 26 03:59:43 ubu-gold systemd[1]: Started Rule-based Manager for Device Events and Files.
Jul 26 03:59:43 ubu-gold systemd[1]: Finished Set the console keyboard layout.
Jul 26 03:59:43 ubu-gold systemd[1]: Reached target Preparation for Local File Systems.
Jul 26 03:59:43 ubu-gold systemd[1]: Mounting Mount unit for core20, revision 2318...
Jul 26 03:59:43 ubu-gold systemd[1]: Mounting Mount unit for lxd, revision 29351...
Jul 26 03:59:43 ubu-gold systemd[1]: Mounting Mount unit for snapd, revision 21759...
Jul 26 03:59:43 ubu-gold systemd[1]: Reached target Mounted snaps.
Jul 26 03:59:43 ubu-gold systemd[1]: Finished Coldplug All udev Devices.
Jul 26 03:59:43 ubu-gold systemd[1]: Condition check resulted in Show Plymouth Boot Screen being skipped.
Jul 26 03:59:43 ubu-gold systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
Jul 26 03:59:43 ubu-gold systemd[1]: Condition check resulted in Forward Password Requests to Plymouth Directory Watch being skipped.
Jul 26 03:59:43 ubu-gold systemd[1]: Reached target Local Encrypted Volumes.
Jul 26 03:59:43 ubu-gold systemd-udevd[376]: Using default interface naming scheme 'v249'.
Jul 26 03:59:43 ubu-gold systemd[1]: Starting Load Kernel Module efi_pstore...
Jul 26 03:59:43 ubu-gold systemd[1]: Condition check resulted in OpenVSwitch configuration for cleanup being skipped.
Jul 26 03:59:43 ubu-gold systemd[1]: Condition check resulted in Show Plymouth Boot Screen being skipped.
Jul 26 03:59:43 ubu-gold systemd[1]: Condition check resulted in Forward Password Requests to Plymouth Directory Watch being skipped.
Jul 26 03:59:43 ubu-gold systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
Jul 26 03:59:43 ubu-gold systemd[1]: Finished Load Kernel Module efi_pstore.
Jul 26 03:59:43 ubu-gold systemd[1]: Condition check resulted in Platform Persistent Storage Archival being skipped.
Jul 26 03:59:43 ubu-gold systemd[1]: Found device /dev/ttyS0.
Jul 26 03:59:43 ubu-gold systemd-udevd[369]: vda: Process '/usr/bin/unshare -m /usr/bin/snap auto-import --mount=/dev/vda' failed with exit code 1.
Jul 26 03:59:43 ubu-gold systemd-udevd[369]: vda15: Process '/usr/bin/unshare -m /usr/bin/snap auto-import --mount=/dev/vda15' failed with exit code 1.
Jul 26 03:59:43 ubu-gold systemd[1]: Found device /dev/disk/by-label/UEFI.
Jul 26 03:59:43 ubu-gold systemd[1]: Starting File System Check on /dev/disk/by-label/UEFI...
Jul 26 03:59:43 ubu-gold systemd-udevd[377]: vda1: Process '/usr/bin/unshare -m /usr/bin/snap auto-import --mount=/dev/vda1' failed with exit code 1.
Jul 26 03:59:43 ubu-gold systemd-fsck[419]: fsck.fat 4.2 (2021-01-31)
Jul 26 03:59:43 ubu-gold systemd-fsck[419]: /dev/vda15: 12 files, 12371/213663 clusters
Jul 26 03:59:43 ubu-gold systemd[1]: Finished File System Check on /dev/disk/by-label/UEFI.
Jul 26 03:59:43 ubu-gold systemd[1]: Mounting /boot/efi...
Jul 26 03:59:43 ubu-gold systemd[1]: Mounted /boot/efi.
Jul 26 03:59:43 ubu-gold systemd[1]: Reached target Local File Systems.
Jul 26 03:59:43 ubu-gold systemd[1]: Starting Load AppArmor profiles...
Jul 26 03:59:43 ubu-gold systemd[1]: Starting Set console font and keymap...
Jul 26 03:59:43 ubu-gold systemd[1]: Starting Create final runtime dir for shutdown pivot root...
Jul 26 03:59:43 ubu-gold systemd[1]: Starting Tell Plymouth To Write Out Runtime Data...
Jul 26 03:59:43 ubu-gold systemd[1]: Starting Set Up Additional Binary Formats...
Jul 26 03:59:43 ubu-gold systemd[1]: Starting Create Volatile Files and Dire
Jul 26 03:59:43 ubu-gold systemd[1]: Mounted Mount unit for core20, revision 2318.
Jul 26 03:59:43 ubu-gold systemd[1]: Mounted Mount unit for snapd, revision 21759.
Jul 26 03:59:43 ubu-gold systemd[1]: Mounted Mount unit for lxd, revision 29351.
...

At first it seemed like an EFI partition problem (for when cloning from template to vm) but below shows all to be well?
Code: 
$lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
loop0     7:0    0   87M  1 loop /snap/lxd/29351
loop1     7:1    0 63.9M  1 loop /snap/core20/2318
loop2     7:2    0 38.8M  1 loop /snap/snapd/21759
sr0      11:0    1    4M  0 rom
vda     252:0    0   32G  0 disk
├─vda1  252:1    0 31.9G  0 part /
├─vda14 252:14   0    4M  0 part
└─vda15 252:15   0  106M  0 part /boot/efi

$ sudo gdisk /dev/vda
GPT fdisk (gdisk) version 1.0.8

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Any pointers on what may be going on, and how to smoothen it out to avoid this hitch would be much appreciated.
 
Last edited:
bbgeek17 said:
Try changing the cloudinit to scsi controller.
Click to expand...
Thank you. Is my interpretation of the comment correct?
Changed drive from virtio0 to scsi0 like so:
Code: 
#qm set ${id} --scsihw virtio-scsi-pci --virtio0 local-lvm:vm-${id}-disk-1,discard=on,size=35020M
qm set ${id} --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-${id}-disk-1,discard=on,size=35020M

Unfortunately, it yields the same result.

1722007412932.png
 
Thank you @bbgeek17... You are a lifesaver! Please do be so kind as to explain why the change of ide to scsi worked. I thought that external CD-ROMs / drives are to be mounted as IDE?
 
Its a bug/compatibility issue between UEFI and IDE cdrom

I thought there was a bugzilla entry for it, but cant find it at this second:
https://forum.proxmox.com/threads/cloud-init-image-only-applies-configuration-on-second-boot.93414/
https://forum.proxmox.com/threads/i...are-not-usable-when-cold-booting-a-vm.117345/

CDroms can be on SCSI bus. I, in fact, had a physical SCSI cdrom many years ago.

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
  • Like
Reactions: morik_proxmox
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back