Hi
@bbgeek17 ,
thanks for your reply.
I will add some background. Researchers can request new Ubuntu VMs with IT support to test stuff and run models and programs and what not. To make this as easy and simple as possible for 1st level support to create new VMs, this process should be completely automated and as fast as possible.
For this reason, I
- downloaded the latest Ubuntu 24.04 Cloud Image as .img
- used virt-customize to install some packages, set a root passwort, ......
- created a new VM, imported the modified .img as scsi0 and did *not* resize it
- added a cloud-init drive as scsi2
- set options like CPU type, scsi controller type, etc
- converted the VM as template
The result: a VM template that can be cloned and started within 1 minute, has all needed base packages and configs in place, and gets it's IP and username/password via cloudinit.
The template VM has a disk size of 3.something GB.
I then set up a html site with an input form, a php site that gets the data, does some checks and calculations, and at the end calls the shell script with the cURL commands. This way the 1st level support can open the page, enter the values like CPU cores, RAM, Disk size, press submit, wait 1 minute, get a random password for the user and the IP back, can give this to the user who requested the VM and be done with it.
Before I went on the website journey, I wrote a script that needed to be run on the PVE node and would interactively ask for the values. It then used pvesh commands to perform all actions.
When this worked, I replaced the interactive questions with the $1, $2, .. variables and called the script with all values as parameters. This worked as well.
I then just rewrote all pvesh commands to a cURL command, leaving the rest of the logic intact. And except of the disk resizing, all other cURL commands worked immediately. This is what lead me to think that I am calling the API endpoint for resizing a disk wrong, which is why...
Since you did not provide actual input values, the low hanging fruit options are:
... I did not provide input values as I thoght the problem to be a syntax error. However, I did test the script with several different values for hard disk size, all larger than 20G.
Example:
Bash:
root@deploy-web:~# bash /usr/local/bin/auto-deploy-v3-web.sh 2 2048 20 user test03
- your "hdd" is not set, or set to wrong value
At the beginning of the script, I echo all parameters, and before every cURL call, I echo what the cURL call does and the values of all used parameters. When I test the script and enter "20" as the desired hdd size, the log line
Bash:
echo "Resizing disk scsi0 to ${hdd}G..."
correctly outputs "Resizing disk scsi0 to 20G..."
So the cURL command should correctly call the resize endpoint to resize the VM disk to 20G.
- your VM does not have scsi0 disk
Since the template does, and cloning the template clones all harddisks, and I have no code that deletes the disk or marks it as unused, the scsi0 disk is definitely there. Additionally, if I use pvesh disk resize after my script finished the resize works, so the cloned VM cannot be the source of the problem.
- you dont have + in your size value, which means the value is taken literally. If your disk is already larger - nothing will happen
https://pve.proxmox.com/pve-docs/api-viewer/#/nodes/{node}/qemu/{vmid}/resize
This is intentional. I have code that sets the size to 5GB if anything smaller than 5GB is entered. If someone requests a VM with 20GB of storage and I use "+", I will end up with a 23.x GB disk, which is not the expected result, so I use the absolute method.
As stated above, the template VM's disk has 3.x GB, so resizing to a minimum of 5GB as an absolute value should always work.
Post actual curl command with all values and remove -s, and provide your VM config
I can craft the cURL commands by hand, as the command in my initial post is the actual command that is called within the script.
Here is my attempt:
Bash:
root@deploy-web:~# ssh root@172.16.10.201 "qm config 999"
root@172.16.10.201's password:
agent: 1
autostart: 1
boot: order=scsi0
ciupgrade: 0
cores: 1
cpu: cputype=x86-64-v2-AES
ipconfig0: ip=dhcp
memory: 1024
meta: creation-qemu=8.1.5,ctime=1721487411
name: ubu2404-ci
nameserver: 8.8.8.8
net0: virtio=BC:24:11:F4:90:EE,bridge=vmbr0
onboot: 1
ostype: l26
scsi0: local-zfs:base-999-disk-0,cache=writeback,discard=on,size=3584M,ssd=1
scsi2: local-zfs:vm-999-cloudinit,media=cdrom
scsihw: virtio-scsi-single
searchdomain: lab.local
smbios1: uuid=a93e201e-09b6-4afb-8b6b-ac850ef7f505
template: 1
vmgenid: 2d0b314d-e356-41d2-b667-1df4080cb110
root@deploy-web:~#
root@deploy-web:~# response=$(curl -s -k -d "username=root@pam&password=password" https://172.16.10.201:8006/api2/json/access/ticket)
root@deploy-web:~# ticket=$(echo $response | jq -r '.data.ticket')
root@deploy-web:~# csrf_token=$(echo $response | jq -r '.data.CSRFPreventionToken')
root@deploy-web:~# echo $ticket
PVE:root@pam:66A2C5BE::LxmGMnYwUv/CxID4b4FJ/HD9tWlGVB/s2yobYvbXNFW4K5HGhHCpih2BiXNd1kVtOJ6xcD7WE0oRre6Wz6qxgigEqRVLuqAQimjZUECOQV2J+jsk1MseyKifJ4+ZuPtwJ77EoxZLADnmMaoBPQniGzR/+4woevG8mZeY++tsLx76HXSYt+LbexjEEUFCKMSC6T2p8X/N7u9orH1EJWQRtOnpYRRLVRe4SJQTn2erfdwTBq/nKcTfKsuQ121quBaMPoNz9V1mT7wY6ZzK2PHxIPTtQABCFQYLQTJE66MWhfV+VtffIkKJtQgTRIzr3kIH+TSuXvS065auK46+8lWu1A==
root@deploy-web:~# echo $csrf_token
66A2C5BE:6zH/vzW6jGFd5k3qXFHXLzGDe2TS0toq+VM56aCwVnE
root@deploy-web:~#
root@deploy-web:~#
root@deploy-web:~# curl -k -b "PVEAuthCookie=$ticket" https://172.16.10.201:8006/api2/json/cluster/nextid
{"data":"1007"}root@deploy-web:~#
root@deploy-web:~#
root@deploy-web:~# curl -k -b "PVEAuthCookie=$ticket" -H "CSRFPreventionToken: $csrf_token" -X POST https://172.16.10.201:8006/api2/json/nodes/pve01/qemu/999/clone -d newid=1007 -d name="forum-test" -d full=1
{"data":"UPID:pve01:000E5805:01D0E7C6:66A2C6E6:qmclone:999:root@pam:"}root@deploy-web:~#
root@deploy-web:~#
root@deploy-web:~#
root@deploy-web:~# curl -k -b "PVEAuthCookie=$ticket" -H "CSRFPreventionToken: $csrf_token" -X POST https://172.16.10.201:8006/api2/json/nodes/pve01/qemu/1007/config -d cores=2 -d memory=2048
{"data":"UPID:pve01:000E5AB3:01D138A6:66A2C7B5:qmconfig:1007:root@pam:"}root@deploy-web:~#
root@deploy-web:~# curl -k -b "PVEAuthCookie=$ticket" -H "CSRFPreventionToken: $csrf_token" -H "Content-Type: application/json" -X POST https://172.16.10.201:8006/api2/json/nodes/pve01/qemu/1007/resize -d disk=scsi0 -d size=20G
root@deploy-web:~#
root@deploy-web:~# curl -k -b "PVEAuthCookie=$ticket" -H "CSRFPreventionToken: $csrf_token" -X POST https://172.16.10.201:8006/api2/json/nodes/pve01/qemu/1007/resize -d disk=scsi0 -d size=20G
{"data":null}root@deploy-web:~#
root@deploy-web:~# ssh root@172.16.10.201 "qm config 1007"
root@172.16.10.201's password:
agent: 1
autostart: 1
boot: order=scsi0
ciupgrade: 0
cores: 2
cpu: cputype=x86-64-v2-AES
ipconfig0: ip=dhcp
memory: 2048
meta: creation-qemu=8.1.5,ctime=1721487411
name: forum-test
nameserver: 8.8.8.8
net0: virtio=BC:24:11:33:52:19,bridge=vmbr0
onboot: 1
ostype: l26
scsi0: local-zfs:vm-1007-disk-0,cache=writeback,discard=on,size=3584M,ssd=1
scsi2: local-zfs:vm-1007-cloudinit,media=cdrom,size=4M
scsihw: virtio-scsi-single
searchdomain: lab.local
smbios1: uuid=563a07ca-97fe-42f4-9e30-af82b6c2f240
vmgenid: 68a9f265-06db-4e03-a91d-0ad749b29ecc
root@deploy-web:~# ssh root@172.16.10.201 "pvesh set nodes/pve01/qemu/1007/resize --disk scsi0 --size 20G"
root@172.16.10.201's password:
UPID:pve01:000E6435:01D269CC:66A2CAC2:resize:1007:root@pam:
root@deploy-web:~# ssh root@172.16.10.201 "qm config 1007"
root@172.16.10.201's password:
agent: 1
autostart: 1
boot: order=scsi0
ciupgrade: 0
cores: 2
cpu: cputype=x86-64-v2-AES
ipconfig0: ip=dhcp
memory: 2048
meta: creation-qemu=8.1.5,ctime=1721487411
name: forum-test
nameserver: 8.8.8.8
net0: virtio=BC:24:11:33:52:19,bridge=vmbr0
onboot: 1
ostype: l26
scsi0: local-zfs:vm-1007-disk-0,cache=writeback,discard=on,size=20G,ssd=1
scsi2: local-zfs:vm-1007-cloudinit,media=cdrom,size=4M
scsihw: virtio-scsi-single
searchdomain: lab.local
smbios1: uuid=563a07ca-97fe-42f4-9e30-af82b6c2f240
vmgenid: 68a9f265-06db-4e03-a91d-0ad749b29ecc
I am at a loss.
To get this project working, I can setup SSH keys and issue the resize command with ssh and pvesh, but in the long term I am curious as to why the cURL call does not work, but the pvesh command does.
