zfs confusion and getting in trouble with qm disk import

[randyqx]

basically, i do not understand zfs image store and am getting in trouble with `qm disk import`. heck, i can not even figure out how to do the the equivalent of `ls -l` to see what is in the zfs store.

i createVM 101 and it creates `images:vm-101-disk-0` as expected. [ i tried to create the VM without allocating a disk, but could not ]. the available space on the store goes down by 34G, the size of the image; which makes sense.

i `qm disk import 101 foo.raw images --format raw`. it reports

Successfully imported disk as 'unused0:images:vm-101-disk-1'

note the "disk-1" and the available space on the store has decreased by another 34G. so i seem to have two images, not what i wanted.

and, of course, the VM instance can not see the imported image. i booted supergrub and grml ISOs and confirmed the VM did not see the imported disk.

 

[bbgeek17]

To list images: pvesm images list
To remove disk : pvesm free images:vm-101-disk-1
To bring disk into VM: qm disk rescan --vmid 100
To make disk available to VM: qm set 100 --scsi[n] unused[n]
To view VM configuration: qm config 100

All of the above, except "disk rescan" can also be achieved via GUI

Good luck

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[randyqx]

clues much appreciated.
aha! as i conjected, it imported into a second image file

# pvesm list images
Volid                Format  Type             Size VMID
images:vm-101-disk-0 raw     images    36507222016 101
images:vm-101-disk-1 raw     images    36507222016 101

rescan seemed happy

# qm disk rescan --vmid 101
rescan volumes...

but, given

# qm config 101
balloon: 1024
boot: order=virtio0;ide2;net0
cores: 1
cpu: qemu64
ide2: local:iso/super_grub2_disk_hybrid_2.04s1.iso,media=cdrom,size=15978K
memory: 2048
meta: creation-qemu=8.1.5,ctime=1716426733
name: foo
net0: virtio=BC:24:11:D1:94:7F,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
smbios1: uuid=624ab1b0-3f17-42f8-856a-93d3ef13a767
sockets: 1
unused0: images:vm-101-disk-1
virtio0: images:vm-101-disk-0,iothread=1,size=34G
vmgenid: a1bbae85-2569-4754-8fc3-803a17efd992

what do i do to either get the VM to mount `images:vm-101-disk-1` as virtio0, or delete the disk-0 and rename disk-1 to disk-0? or maybe i am expected to do something else.

 

[bbgeek17]

what do i do to either get the VM to mount `images:vm-101-disk-1` as virtio0, or delete the disk-0

To remove disk-0
"man qm"

qm disk unlink <vmid> --idlist <string> [OPTIONS]

       Unlink/delete disk images.

       <vmid>: <integer> (100 - 999999999)
           The (unique) ID of the VM.

       --force <boolean>
           Force physical removal. Without this, we simple remove the disk from the config file and create an additional configuration entry called unused[n], which contains the volume ID. Unlink of unused[n] always cause
           physical removal.

       --idlist <string>
           A list of disk IDs you want to delete.

To set disk-1 as virtio0:
"man qm" > qm set --virtio0

There is no point in renaming disk-1 to disk-0, its just backend indexing.

Good luck

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[randyqx]

earlier i tried to hack around there a little and did not succeed. clearly, among many other things, i do not know what a "disk ID" is, clearly it is not a VolID

# qm disk unlink 101 --idlist images:vm-101-disk-0
400 Parameter verification failed.
idlist: invalid format - invalid configuration ID 'images:vm-101-disk-0'

qm disk unlink <vmid> --idlist <string> [OPTIONS]
 
# qm disk unlink 101 --idlist vm-101-disk-0
400 Parameter verification failed.
delete: unknown option 'vm-101-disk-0'
qm disk unlink <vmid> --idlist <string> [OPTIONS]

# qm disk unlink 101 --idlist virtio0
update VM 101: -delete virtio0
400 Parameter verification failed.
virtio0: hotplug problem - can't unplug bootdisk 'virtio0'

qm disk unlink <vmid> --idlist <string> [OPTIONS]

 

[bbgeek17]

# qm disk unlink 101 --idlist virtio0 update VM 101: -delete virtio0 400 Parameter verification failed. virtio0: hotplug problem - can't unplug bootdisk 'virtio0'

You need to stop the VM:
qm stop 101

removing boot disk on live VM is a bit "tricky".

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[randyqx]

i think it is

# qm status 101
status: stopped

waited five minutes, tried again, and it worked!

 

[bbgeek17]

Works for me:

root@pve-1:~# pvesm alloc blockbridge-iscsi 100 vm-100-disk-11 1G
successfully created 'blockbridge-iscsi:vm-100-disk-11'
root@pve-1:~# qm set 100 --virtio0 blockbridge-iscsi:vm-100-disk-11
update VM 100: -virtio0 blockbridge-iscsi:vm-100-disk-11
root@pve-1:~# qm disk unlink 100 --idlist virtio0
update VM 100: -delete virtio0

root@pve-1:~# qm set 100 --virtio0 blockbridge-iscsi:vm-100-disk-11
update VM 100: -virtio0 blockbridge-iscsi:vm-100-disk-11
root@pve-1:~# qm set 100 --boot c --bootdisk virtio0
update VM 100: -boot c -bootdisk virtio0
root@pve-1:~# qm disk unlink 100 --idlist virtio0
update VM 100: -delete virtio0

unless the VM was running at the time:

root@pve-1:~# pvesm alloc blockbridge-iscsi 100 vm-100-disk-10 1G
successfully created 'blockbridge-iscsi:vm-100-disk-10'
root@pve-1:~# qm set 100 --virtio0 blockbridge-iscsi:vm-100-disk-10
update VM 100: -virtio0 blockbridge-iscsi:vm-100-disk-10
root@pve-1:~# qm set 100 --boot c --bootdisk virtio0
update VM 100: -boot c -bootdisk virtio0
root@pve-1:~# qm start 100
generating cloud-init ISO
root@pve-1:~# qm disk unlink 100 --idlist virtio0
update VM 100: -delete virtio0
400 Parameter verification failed.
virtio0: hotplug problem - can't unplug bootdisk 'virtio0'

qm disk unlink <vmid> --idlist <string> [OPTIONS]

However, the change is pending and on stop the disk unlink is completed:

root@pve-1:~# qm stop 100
root@pve-1:~# qm disk unlink 100 --idlist virtio0
update VM 100: -delete virtio0
cannot delete 'virtio0' - not set in current configuration!

note, I did not use --force. So I still need to delete "unusedX"

Good luck

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[randyqx]

to close the thread, at least for me. what worked for me

#!/bin/sh
# do-create vmid memK name
qm create \
    $1 \
    --autostart=1 \
    --boot order=virtio0 \
    --memory=$2 \
    --name=$3 \
    --net0 virtio,bridge=vmbr0,firewall=1

to create a vm with no disk. and then import the file `dd`ed over from ganeti instance's `/dev/ganeti/primaryblarble.disk0_data`

#!/bin/sh
# do-import vmID file
qm disk import \
    $1 \
    $2 \
    images \
    --format raw
qm set $1 --virtio0 file=images:vm-$1-disk-0
# beware, interface ID within `/etc/network/interfaces` is likely
#         to have changed :(

 

[alexskysilk]

not proxmox specific, but make it a habit to put all variables in quotes- or you WILL end up getting bit by unexpected variable expansion.

 

[randyqx]

yes, i have lazy shell habits, picked up over decades. but reminders appreciated.