QCOW2 on ZFS, but over NFS?

[IsThisThingOn]

So you don't have a pve cluster with zfs replication between running ?

Not sure how this is relevent. But since you ask; no, I don't believe in clusters for Proxmox

HA is IMHO grossly overrated and in many setups it causes more downtime because of added complexity than it actually saves uptime.
Especially considering how rock solid a good server with ZFS mirrors and redundant PSU runs.
Even for single nodes, software has always be a way higher risk factor than hardware for me. To be honest, in all these decades running servers, I never had a single downtime because of hardware. I had once a PSU failing, once DELL VRTX CMC, 3 APC batteries and maybe 10 disks failing, not a single failure that actually caused downtime (besides the two times fiber line cutting from construction and once power outage for 3 hours, but these would have happened with a cluster too).

 

[guruevi]

Basically for snapshots to work transparently you have to write a plugin for your storage system. Could be a simple and gross as “ssh root@storage zfs snapshot volume” but there needs to be some integration between your storage and your VM system. Some vendors already made those plugins, Proxmox has a few examples.

 

[alexskysilk]

Could be a simple and gross as “ssh root@storage zfs snapshot volume”

ahhh but thats only the beginning of the pain.

you also need to coordinate guest quiescence (freeze/thaw) with appropriate state detection, write the snapshot state data into vmid,conf, snapshot chain management, restoration logic, control iscsi lun mapping, etc etc.

It is doable if the storage has an api exposed- ssh isnt commonly offered on a storage product but if its a linux/freebsd/openindiana type device you may even be able to use the built in zfs-over-iscsi plugins already provided. I had considered creating a plugin for QNAP QUTS but got deep enough the rabbithole to glimpse just how much work is involved- I backed away VERY quickly I dont have that kind of free time.

 

[guruevi]

you also need to coordinate guest quiescence (freeze/thaw)

that is exactly what Proxmox does, the plugin just calls whatever code you want between freeze and thaw