Making linked clones from snapshots of VMs (NOT TEMPLATES)

[pato_ehm]

Hi,

currently we are evaluating Proxmox as a replacement for vSphere. In our usual setup, we work with master VMs that are used to create linked clones from snapshots of these master VMs.

This feature is important for us, as our workflow would become much more complex and costly if we had to rely solely on templates.

I was already able to achieve this in Proxmox when using Ceph. However, this was not on ZFS, which is what we are planning to use on the final server.

AI tools were not able to provide further guidance on how to enable this with ZFS. Can someone help me with this? I would mainly like to know whether this is even possible with ZFS and, if so, under which configuration.


This is what i tried:

root@<server-name>:~# qm  clone 107 1071 --name linked-clone-1071 --snapname snapshot-1  --full 0
Linked clone feature is not supported for a snapshot of 'local-zfs:vm-107-disk-1' (efidisk0)
root@<server-name>:~# qm config 107
agent: 1
bios: ovmf
boot: order=ide0;ide1;sata0
cores: 1
cpu: host
efidisk0: local-zfs:vm-107-disk-1,efitype=4m,size=1M
ide0: none,media=cdrom
ide1: none,media=cdrom
machine: pc-i440fx-11.0
memory: 65536
meta: creation-qemu=11.0.0,ctime=1781098982
name: MvmW11Proxmox
net0: vmxnet3=00:50:56:85:36:05,bridge=VLAN61
ostype: win10
parent: snapshot-1
sata0: local-zfs:vm-107-disk-0,size=2T
scsihw: virtio-scsi-single
serial0: socket
smbios1: uuid=420534a0-e064-1795-9be7-94f24a53cb22
sockets: 4
template: 0
vmgenid: c775a5e3-cc9f-4a63-8b5c-91335bd56453
root@<server-name>:~# qm set 107 --bios seabios
update VM 107: -bios seabios
root@<server-name>:~# qm set 107 --delete efidisk0
update VM 107: -delete efidisk0
root@<server-name>:~# qm snapshot 107 snapshot-1
snapshotting 'drive-sata0' (local-zfs:vm-107-disk-0)
root@<server-name>:~# qm  clone 107 1071 --name linked-clone-1071 --snapname snapshot-1  --full 0
Linked clone feature is not supported for a snapshot of 'local-zfs:vm-107-disk-0' (sata0)
root@<server-name>:~# qm set 107 --scsi0 local-zfs:vm-107-disk-0
update VM 107: -scsi0 local-zfs:vm-107-disk-0
root@<server-name>:~# qm set 107 --delete sata0qm set 107 --delete sata0qm set 107 --delete sata0^C
root@<server-name>:~# ^C
root@<server-name>:~# ^C
root@<server-name>:~# ^C
root@<server-name>:~# ^C
root@<server-name>:~# ^C
root@<server-name>:~# qm set 107 --delete sata0
update VM 107: -delete sata0
root@<server-name>:~# qm delsnapshot 107 snapshot-1
root@<server-name>:~# qm snapshot 107 snapshot-1
snapshotting 'drive-scsi0' (local-zfs:vm-107-disk-0)
root@<server-name>:~# qm  clone 107 1071 --name linked-clone-1071 --snapname snapshot-1  --full 0
Linked clone feature is not supported for a snapshot of 'local-zfs:vm-107-disk-0' (scsi0)

 

[LnxBil]

root@<server-name>:~# qm clone 107 1071 --name linked-clone-1071 --snapname snapshot-1 --full 0
Linked clone feature is not supported for a snapshot of 'local-zfs:vm-107-disk-1' (efidisk0)

AFAIK, this is a (hopefully) temporary limitation that has been worked on in the past and it only concerns the EFI disk.
You can always create a cloned VM manually yourself, if you build a new VM, clone the data disk and create a new EFI disk. This may be a workaround.

I was already able to achieve this in Proxmox when using Ceph. However, this was not on ZFS, which is what we are planning to use on the final server.

Why are you testing with an superior technology just to go with an inferior one? I love ZFS, but it's not for a cluster setup.
Another approach would be single-node-CEPH, which works and would also solve your problem.

 

[pato_ehm]

AFAIK, this is a (hopefully) temporary limitation that has been worked on in the past and it only concerns the EFI disk.
You can always create a cloned VM manually yourself, if you build a new VM, clone the data disk and create a new EFI disk. This may be a workaround.

Well yes, but this would be making a full clone right?

Why are you testing with an superior technology just to go with an inferior one? I love ZFS, but it's not for a cluster setup.
Another approach would be single-node-CEPH, which works and would also solve your problem.

My understanding was that Ceph must have at least 3 Nodes. It would be then possible to achieve this with one server only? We would have only one server with mutpile storages that we planned to use with a zfs pool. Would you recommend a single-node-CEPH for this setup? We would be using it only for the creation of linked clones from Master VMs.

 

[LnxBil]

Well yes, but this would be making a full clone right?

If you do that yes, if you don't it won't. You have to do it manually with zfs clone and that is just a reference clone, but you need to do this yourself, e.g. with ansible or any other tooling of your choice:

• create a VM, this will create it without disks
• attach an efi disk
• clone your reference os/data disk by hand to the correct name (similar to the old one, but with the correct id)
• attach the cloned disk

then you're done.

My understanding was that Ceph must have at least 3 Nodes. It would be then possible to achieve this with one server only? We would have only one server with mutpile storages that we planned to use with a zfs pool. Would you recommend a single-node-CEPH for this setup? We would be using it only for the creation of linked clones from Master VMs.

Sure, but a single-node-ceph has the same availability as a single-ZFS node, so you won't gain or loose anything. It's just a solution to your clone-problem.
The computational and memory impact is larger due to the more processes running and more layered I/O path, but it will just work as it should with the clone.

Going full CEPH (3-node) is no option?

 

[fiona]

For reference: https://bugzilla.proxmox.com/show_bug.cgi?id=7521

 

[pato_ehm]

If you do that yes, if you don't it won't. You have to do it manually with zfs clone and that is just a reference clone, but you need to do this yourself, e.g. with ansible or any other tooling of your choice:

• create a VM, this will create it without disks
• attach an efi disk
• clone your reference os/data disk by hand to the correct name (similar to the old one, but with the correct id)
• attach the cloned disk

then you're done.



Sure, but a single-node-ceph has the same availability as a single-ZFS node, so you won't gain or loose anything. It's just a solution to your clone-problem.
The computational and memory impact is larger due to the more processes running and more layered I/O path, but it will just work as it should with the clone.

Going full CEPH (3-node) is no option?

Thanks! the first approach worked pretty well