Cloud-init on ubuntu NOT using ubuntu cloud images.

[Tim Gildersleeve]

Hi,

I've been playing with cloud-init initially using the cloud images provided by Ubuntu. Works very well and great way to create faster images by having them on the smaller SSD due to the VMs being linked.

I've been following https://pve.proxmox.com/wiki/Cloud-Init_Support and its worked great for the cloud images.

However, I want to create my templates with some default options, default packages etc so I decided to create my own image. Starting with a Ubuntu 1804 server install I set it up how I want it, didnt need to install cloud-init as Ubuntu 1804 server installs this by default regardless.

Shut the server down and then added a cloud-init device as directed. Converted to template and then created a linked clone. Edited the cloud-init settings on this new VM, booted and logged in (using the configured user I created on the template). However the networking has not been set by cloud-init. I then tried changing the password via cloud-init (I know not recommended as it is less secure than using keys) and that didnt work either - the original password was still used.

I mounted the CDROM and checked the settings there - and they are as I set them in the cloud-init setup in the proxmox so the problem seems to be that cloud-init is not using the cdrom as its data store.

Can anyone give me a push in the right direction as to how to get this to work? I am sure I am missing something obvious - but after spending several hours looking for information on this I have no idea what it is!

Regards

Tim

 

[tom]

See also:

https://pve.proxmox.com/wiki/Cloud-Init_FAQ
https://pve.proxmox.com/wiki/Cloud-Init_Support

 

[Tim Gildersleeve]

Thanks Tom,

Not sure why I missed the FAQ but I found my problem there. Had to search around a bit and follow links around but finally found my problems were:

Not adding a cfg file for pve to select the data sources (although I had done a dpkg-reconfigure cloud-init and selected those sources so that may not have been an issue.

Predictable network devices. This was the biggie - not having an eth0 was causing the issues so I have reverted to the old style interface names for now and this seems to work.

Ubuntu 18.04 from Ubuntu's installation sources: There seems (to me) to be something strange that Ubuntu 18.04 installation does when it installs cloud-init (preinstalled without asking on 18.04 server). Whatever I tried (and I know there must have been something I missed that would have fixed it) I could not get the cloud-init working on 18.04. Installing Ubuntu 17.10 and then upgrading to 18.04 and then installing cloud-init worked perfectly once I had the network interface names sorted.

In case anyone else sees this and has problems it seems the biggest gotcha is predictable network names - turn them off! Follow the links in the FAQ Tom linked above for details on predictable network names (would link them myself but cant post links!)

Thanks for you help Tom and hope this helps someone else!

Regards

Tim

 

[fragan]

Hi everybody,

I would like to share the result as I had the same problem of Tim. So for me, when I was using the official Ubuntu cloudimg and follow Proxmox https://pve.proxmox.com/wiki/Cloud-Init_Support wiki, I didn't have any problem. Then like Tim when I wanted to build my own custom image, but it was not working, and I didn't want to install ubuntu 17.10.
So I will not repeat everything but yes you have to change the network interface name problem as described. but also some changes in the grub configuration, so those following files should look like that (I especially keep in comment the original parameters) :
- /etc/default/grub
"
GRUB_DEFAULT=0
GRUB_HIDDEN_TIMEOUT=0
GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
#GRUB_CMDLINE_LINUX_DEFAULT="maybe-ubiquity"
GRUB_CMDLINE_LINUX_DEFAULT="console=tty1 console=ttyS0"
#GRUB_CMDLINE_LINUX=""
GRUB_CMDLINE_LINUX="net.ifnames=0 biosdevname=0"
"
- /etc/default/grub.d/50-curtin-settings.cfg
"
#GRUB_CMDLINE_LINUX_DEFAULT="maybe-ubiquity"
# disable grub os prober that might find other OS installs.
#GRUB_DISABLE_OS_PROBER=true
GRUB_RECORDFAIL_TIMEOUT=0
GRUB_TIMEOUT=0
GRUB_TERMINAL=console
"
then launch update-grub.

after that I run dpkg -reconfigure cloud-init like Tim, and you should disable everything and keep NoCloud, ConfigDrive , or edit the file /etc/cloud/cloud.cfg.d/99_pve.cfg as describe in Promox wiki but delete the orignal file 90_dpkg.cfg.

The problem was the system didn't recognize the serial interface, I understood by running "cloud-init analyze show" look the end result of this command (I deleted some lines) :
"
-- Boot Record 31 --
Starting stage: init-local
|`->restored from checked cache: DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net] @00.00700s +00.01200s
Finished stage: (init-local) 00.10700 seconds
Starting stage: init-network
|`->restored from cache with run check: DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net] @00.66400s +00.00300s
"

I resolve this by running "sudo cloud-init clean" and magic (/dev/sr0) appears :

"
-- Boot Record 32 --
Starting stage: init-local
|`->no cache found @00.00900s +00.00100s
|`->found local data from DataSourceNoCloud @00.02800s +00.12100s
Finished stage: (init-local) 00.41800 seconds
Starting stage: init-network
|`->restored from cache with run check: DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net,/dev/sr0][dsmode=net] @02.06000s +00.00400s

I which it will help someone else, English is not my mother languages so apologize for mistakes.

 

[Jacolex]

Hello
I found that problem is with cloud-init configuration which is generated by Ubuntu 18.04 installer. I spent many hours and I finally resolve the issue, doing this:

apt remove cloud-init
apt purge cloud-init
rm -rf /var/lib/cloud/
rm -rf /etc/cloud/
apt install cloud-init
​

After that the initial configuration is deleted and then cloud-init default config files are generated, so proxmox config files from sr0 with networking will be successfully processed. No grub modification needed.
For example, template has ens18 interface, and after cloud-init processing it's renamed to eth0:
network:
version: 2
ethernets:
eth0:
addresses:
- 172.16.112.107/20
gateway4: 172.16.127.250
match:
macaddress: ba:37:39:83:7f:f9
nameservers:
addresses:
- 8.8.8.8
search: []
set-name: eth0


Of course, for compatibility reasons I will modify grub of the template to use legacy interface names to avoid problem with various naming conventions in one system.

 

[NoahD]

Another thing you may want to is remove the machine-id in /etc/machine-id so it'll be unique in each clone generation.

cat /etc/machine-id
rm /etc/machine-id
touch /etc/machine-id

I don't know if cloud-init does that automatically but least doing this wouldn't hurt before converting the image as a template.

EDIT: I just tested it. Cloud-init does NOT regenerate a new machine-ID so best to remove the machine-id in the file before turning the image into a template.

 

[rsmvdl]

Another thing you may want to is remove the machine-id in /etc/machine-id so it'll be unique in each clone generation.

cat /etc/machine-id
rm /etc/machine-id
touch /etc/machine-id

I don't know if cloud-init does that automatically but least doing this wouldn't hurt before converting the image as a template.

EDIT: I just tested it. Cloud-init does NOT regenerate a new machine-ID so best to remove the machine-id in the file before turning the image into a template.

dude, you have to do the following:

rm /etc/machine-id
touch /etc/machine-id
rm /var/lib/dbus/machine-id
ln -s /etc/machine-id /var/lib/dbus/machine-id

in order to make generate a new machine ID working.

 

[klowet]

Hi

If it can help, I had the same problem with 18.04.3 with the network.
I started with the Ubuntu mini.iso (only selected 'SSH-server' during the install) and when cloning my own image, the network wouldn't come online until a reboot.

All I had to do in my image to solve this problem was: systemctl enable systemd-networkd
When you start from mini.iso and only installs SSH-server, don't forget to install cloud-init and qemu-guest-agent (and enable qemu-guest-agent).
After that, the network came online immediately after cloning. No need for changing the grub file for adding a tty-serial, neither changing interface names from ens to eth.

// edit
I just tried with the netinstall ISO and selected the "Ubuntu cloud image" during install. That works too after enabling systemd-networkd. Reinstalling cloud-init or removing the default config files isn't necessary.