New VM from Cloud-Init Image via API?

[utkonos]

Using a cloud image to create a new VM using the command line is straightforward. I'm running into a problem using the API to do the same.

I am using this cloud image:
https://cloud-images.ubuntu.com/releases/22.04/release/ubuntu-22.04-server-cloudimg-amd64.img

I'm using this API endpoint (substituting node for my node's name):
/api2/json/nodes/{node}/qemu

And then the following config:

{
        'name': 'test',
        'vmid': 101,
        'ostype': 'l26',
        'memory': 2048,
        'cores': 4,
        'scsihw': 'virtio-scsi-pci',
        'virtio0': 'local-zfs:0,import-from=local:iso/ubuntu-22.04-server-cloudimg-amd64.img,backup=0,format=qcow2',
        'cipassword': 'password',
        'boot': 'order=virtio0',
        'net0': f'model=virtio,bridge=vmbr1'
}

I'm getting a 400 error:


local:iso/ubuntu-22.04-server-cloudimg-amd64.img has wrong type 'iso' - not an image

 

[bbgeek17]

"image" in this context is a PVE volume, it probably thinks the file is ISO because its in ISO storage..

I have not used import-from in combined in VM create, the man page says it imports-from a "volume" specifically. Try to create a volume first, i.e. "qm importdisk $VMID $ISO/$OSIMAGE $STORAGE --format raw" and then use that volume, i.e. two steps.
Based on the wording of the man page and API doc I am not sure it should work as you expect it today.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[utkonos]

Thanks, my goal is not to have to use qm command line at all. I want to start with nothing and then use a set of API calls to end up with a Ubuntu VM that I can SSH into (password or ssh key: both are acceptable for the initial stage).

FreeBSD is easy. Windows is easy. Ubuntu so far is not straightforward.

 

[utkonos]

I think this may be the answer.

https://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html

ug. or maybe not.

 

[bbgeek17]

I wasnt saying to use qm, but that you may have to split your request into to APIs. But it looks like thats not the issue.
The problem is that import-from, like import, is not wired to use storage objects like "local" for file image import. It only takes full path at the moment:

+ qm create 206 --memory 512 --name vm206 --socket 1 --onboot no --scsi0 blockbridge1:0,import-from=bbnas:iso/cirros-0.5.1-x86_64-disk.img
400 Parameter verification failed.
scsi0: bbnas:iso/cirros-0.5.1-x86_64-disk.img has wrong type 'iso' - not an image
qm create <vmid> [OPTIONS]

+ qm create 207 --memory 512 --name vm207 --socket 1 --onboot no --scsi0 blockbridge1:0,import-from=/mnt/pve/bbnas/template/iso/cirros-0.5.1-x86_64-disk.img
transferred 0.0 B of 112.0 MiB (0.00%)

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[utkonos]

Thanks for the assistance, as always!

I went in a different direction. It is amazing how complex the various parts of the documentation and howtos out there make this. However, I boiled it all down to an extremely simple 37 line Python script which takes the Ubuntu server 22.04 LTS install ISO and edits it. The grub.cfg is edited to perform autoinstall and the two files, user-data and meta-data are added. A new ISO is written based on the edits. The resulting ISO is then uploaded to ISO storage on Proxmox and a new VM is deployed from that ISO image. Clean as a whistle and works.

Here is that script for anyone else who needs it:

https://gist.github.com/utkonos/718b150de4f86054c37ac798c02b54c6

 

[dermorz]

The problem is that import-from, like import, is not wired to use storage objects like "local" for file image import. It only takes full path at the moment:

+ qm create 207 --memory 512 --name vm207 --socket 1 --onboot no --scsi0 blockbridge1:0,import-from=/mnt/pve/bbnas/template/iso/cirros-0.5.1-x86_64-disk.img
transferred 0.0 B of 112.0 MiB (0.00%)

Is there a way to get the (local) full path of disk images via the API, so the import-from flag can actually be used from outside? Or is there a reliable way to compose it from the info I gather from /api2/json/nodes/{node}/storage and /api2/json/nodes/{node}/storage/{storage}/content?

/edit:
After reading that the process of creating VMs with cloud images is creating a VM without any disks and then importing and adding cloud images as disks before booting, my question seems to boil down to: Is qm importdisk ... available via the API?

/edit2:
I found https://forum.proxmox.com/threads/import-disk-from-file-using-api.93393/ which answers my question. The endpoint I need (and probably many others) is not implemented yet. Any update on that end @Fabian_E?

 

[utkonos]

I found https://forum.proxmox.com/threads/import-disk-from-file-using-api.93393/ which answers my question. The endpoint I need (and probably many others) is not implemented yet.

That is correct. The API endpoint needed to automate this task does not exist (yet?). I burned a support ticket and learned this fact.

 

[fiona]

Hi,

I wasnt saying to use qm, but that you may have to split your request into to APIs. But it looks like thats not the issue.
The problem is that import-from, like import, is not wired to use storage objects like "local" for file image import. It only takes full path at the moment:

This is not entirely true...

+ qm create 206 --memory 512 --name vm206 --socket 1 --onboot no --scsi0 blockbridge1:0,import-from=bbnas:iso/cirros-0.5.1-x86_64-disk.img
400 Parameter verification failed.
scsi0: bbnas:iso/cirros-0.5.1-x86_64-disk.img has wrong type 'iso' - not an image

...the real issue is that the content type here is iso rather than images. You can use storage syntax when the content type is correct. Feel free to open an enhancement request to enable importing from content type iso too.

qm create <vmid> [OPTIONS]

+ qm create 207 --memory 512 --name vm207 --socket 1 --onboot no --scsi0 blockbridge1:0,import-from=/mnt/pve/bbnas/template/iso/cirros-0.5.1-x86_64-disk.img
transferred 0.0 B of 112.0 MiB (0.00%)

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[fiona]

Hi,

Is there a way to get the (local) full path of disk images via the API, so the import-from flag can actually be used from outside? Or is there a reliable way to compose it from the info I gather from /api2/json/nodes/{node}/storage and /api2/json/nodes/{node}/storage/{storage}/content?

/edit:
After reading that the process of creating VMs with cloud images is creating a VM without any disks and then importing and adding cloud images as disks before booting, my question seems to boil down to: Is qm importdisk ... available via the API?

/edit2:
I found https://forum.proxmox.com/threads/import-disk-from-file-using-api.93393/ which answers my question. The endpoint I need (and probably many others) is not implemented yet. Any update on that end @Fabian_E?

I answered the thread there and included an example which uses import-from to import from an existing image on a storage (no need to know the full path then).

 

[dermorz]

...the real issue is that the content type here is iso rather than images. You can use storage syntax when the content type is correct. Feel free to open an enhancement request to enable importing from content type iso too.

I think there is some kind of confusion here. The file type of the image is definitely correct, but proxmox treats it as iso. Probably because there is only the possibility to upload "iso" files and it always sets this content-type?

curl -XPOST --silent -v --insecure -H "Authorization: PVEAPIToken=$(cat ./proxmox-api-token.txt)" \
-H "Content-Type: application/json" \
--data '{"vmid": 102, "scsihw": "virtio-scsi-pci", "virtio0": "local:0,import-from=local:iso/ubuntu-server-cloudimg-amd64.img", "agent": 1, "serial0": "socket"}' \
https://192.168.121.209:8006/api2/json/nodes/bullseye/qemu

With:

root@bullseye:~# file /var/lib/vz/template/iso/ubuntu-server-cloudimg-amd64.img
/var/lib/vz/template/iso/ubuntu-server-cloudimg-amd64.img: QEMU QCOW2 Image (v2), 2361393152 bytes

Gives me:

{"data":null,"errors":{"virtio0":"local:iso/ubuntu-server-cloudimg-amd64.img has wrong type 'iso' - not an image"}}

In the example you give on the other thread, you are not importing an uploaded image file as disk but the disk of another machine:

pvesh set /nodes/pve701/qemu/105/config --scsi1 local:0,discard=on,import-from=myzpool:vm-123-disk-0
...
create full clone of drive scsi0 (myzpool:vm-123-disk-0)
...
scsi1: successfully created disk 'local:105/vm-105-disk-1.raw,discard=on,size=103M'

So if I interpret your example and what you wrote correctly, it is only possible to use import-from= with other disk images, not uploaded qcow2 images. Since I need an existing VM or template anyways to achieve that, I can go the template path all the way like desribed here: https://yetiops.net/posts/proxmox-terraform-cloudinit-saltstack-prometheus/#creating-a-template

 

[bbgeek17]

@Fabian_E , I think I see where the confusion is:
There is no dedicated place in Proxmox for qcow/raw/etc type pre-built images (really templates), like the ones provided by every Linux distro for cloud deployments.

When users, me included, are looking where to place those - the logical place seems to be /templates/iso. However, Proxmox is hard coded to treat those as ISO regardless of their actual type when used through API/CLI and via storage object (i.e. not absolute path).
The only other place that is dedicated to images, as described here https://pve.proxmox.com/wiki/Storage:_Directory, is /images/[vmid]/ . And the files have to be properly named - loosing their naming metadata.

So, essentially, even though many treat ISO location as a sort of "Openstack Glance" - its not. The solution, imho, is not to enhance product to "import iso". It should be an official /template/images location where qcows/raws/etc can be placed for import.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[utkonos]

@bbgeek17 Precisely. And thanks for stating it so clearly. My goal is to have a piece of code that lives outside of Proxmox (or running on a VM in Proxmox, but the effect would be external. That code will have two pathways:

A. For cloud images where the OS provides compressed images like FreeBSD:
1. Download from that location: https://download.freebsd.org/ftp/releases/VM-IMAGES/13.1-RELEASE/amd64/Latest/
2. Decompress the image.
3. Verify the image using GPG and SHA hashes as appropriate.
3. Send the image and its hash to a location on Proxmox using an call to an API endpoint (this is the missing component in the API as far as I understand).

B. For cloud images where the OS provides uncompressed images like Ubuntu:
1. Call an API endpoint on Proxmox that provides the download URL and hash gleaned from here along with GPG checks as appropriate: https://cloud-images.ubuntu.com/daily/server/jammy/current/
2. The API endpoint then downloads the cloud image to the correct location and verifies the hash.

Then there is a "finally" clause (in Pythonese) after all the try/except/else's above:

Finally: run a script that instantiates and configures N VMs based on that "template" or "cloud image" however it is best referred to.

But as I said earlier, I don't need this anymore. It would just be a nice to have. I have abandoned using distro provided cloud images altogether. I am using the unattended installation features of each OS along with an ISO editing python package (pycdlib) to create a custom ISOs that I then use as the starting point for VM creation. This process has the same end result that I want.

 

[fiona]

Thank you for explaining your use cases! I went ahead and created an enhancement request to better keep track of the issue.

 

[CarlFK]

I want this too.

Does someone have a launchpad PPA that can host a package with this/these patches? If so, I'll install it on my crash test box and test/use it. for this I'm pretty sure if the resulting VM seems good, I can move it to my production box and feel pretty good about everything. Or if there seem to be problems, I'll open issues.

I could patch/build from source, but I don't do that often enough to expect it to go smooth and I'd rather not open an issue just to waste someone elses time figuring out that I didn't patch/build as expected.

 

[fiona]

Hi,

I want this too.

Does someone have a launchpad PPA that can host a package with this/these patches? If so, I'll install it on my crash test box and test/use it. for this I'm pretty sure if the resulting VM seems good, I can move it to my production box and feel pretty good about everything. Or if there seem to be problems, I'll open issues.

I could patch/build from source, but I don't do that often enough to expect it to go smooth and I'd rather not open an issue just to waste someone elses time figuring out that I didn't patch/build as expected.

there is recent work being done for OVF/OVA import: https://lists.proxmox.com/pipermail/pve-devel/2024-April/063419.html Can't give any guarantees, but it's likely to land in the following ~weeks/months.