Non-VM-specific disks

[cape]

Hi,
I wonder if there is a way to have disks not specific to a certain VM, but possible to attach/detach to different VMs?
I found this documentation for migrating disks between VMs, but it is fairly involved and much of it seems not meant for automation: https://pve.proxmox.com/wiki/Moving_disk_image_from_one_KVM_machine_to_another

For context, I'm investigating how hard it would be to build a Kubernetes storage driver for Proxmox, so containers would be scheduled on different VMs.

 

[wolfgang]

Hi,

in Proxmox VE storage model a VM image is owned by a VM.

 

[cape]

Hi Wolfgang,
Thanks for your reply. This also includes non-root disks, then?
Also, in case I wasn't clear, I don't want to attach the disk to multiple VMs at the same time, only be able to move it around between VMs (with an intermediate "attached to no VM at all" state).

I did a bit of poking on the API, and it seems I can actually attach a disk to a VM which does not own it (actually, the "owner" VM doesn't even exist):

pvesm alloc local-lvm 2000 vm-2000-misc-disk 1G
pvesh set /nodes/prx/qemu/101/config -virtio1=local-lvm:vm-2000-misc-disk
<ssh to VM 101>
$ dmesg
...
[384414.925462] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)
[384414.926170] virtio_blk virtio4: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)

So it at least sort of works... Would there be any issues with this approach (aside from ensuring the magic VM id 2000 in this example is not used...)?

 

[wolfgang]

This also includes non-root disks, then?

Yes, this is why Proxmox VE encodes the images in this schema <type>-<VMID>-disk-<NO>

So it at least sort of works... Would there be any issues with this approach (aside from ensuring the magic VM id 2000 in this example is not used...)?

Proxmox VE has a function "[pm|pct] rescan" which scans the storage and add images as unused to the owner guest if they are not referenced in the owner guest config.
So if an image is referends in another config the recan will add it in the owner config too.

 

[cape]

Proxmox VE has a function "[pm|pct] rescan" which scans the storage and add images as unused to the owner guest if they are not referenced in the owner guest config.
So if an image is referends in another config the recan will add it in the owner config too.

Sorry, this wasn't clear to me. You mean if there is in fact a VM with that "magic id" (I chose 2000 in my example), and the user enters that command, the disk will be attached to that VM as well? Would it be forcefully detached then from the previously attached-to VM? Or (worse) would it be attached to _both_ VMs?

Am I understanding it correctly that there isn't really a "good" way to accomplish this, and it realistically won't be possible in the near term at least? Or would my "workaround" be considered acceptable to recommend?

 

[wolfgang]

Would it be forcefully detached then from the previously attached-to VM?

No, because in our model we assume he is the owner and so there is no need for extra checks.

Or (worse) would it be attached to _both_ VMs?

It will add in the owner VM as an unused disk, so the users have to active attach the disk.
But if he does this there is no check if the disk is used in more then one VM.
If so this will probably end in data corruption.

Am I understanding it correctly that there isn't really a "good" way to accomplish this, and it realistically won't be possible in the near term at least? Or would my "workaround" be considered acceptable to recommend?

I think the only clean way would be to rename the disk. But there are no plans to implement this.
But you can discuss this on the dev list.