I would like to replicate zfs filesystems (datasets) between nodes in a Proxmox cluster, which can certainly be done if a VM or Container is parked on it. It can also certainly be done from command line with zfs send / zfs recv. What can't be done is to be able to do this through the GUI, why is that? Or why as an alternative. and probably a better solution, can't you have a container mount a dataset though a bind mount and replicate it that way?
The first solution has its drawbacks in that it wouldn't be clear which node has the most up to date copy as a replication source, but in the second solution the node the container is running on could be the designated master. All "regular" VMs and containers could simply mount as NFS or CIFS and always be on the same version of the data. This way some nodes could serve as network storage with HA serving their shares to other compute only nodes or even to each other.
While I can kind of understand why there are no provisions for the first scenerio, as you really don't know who has the authoritive copy, in the 2nd scenerio you do, or at least as much so as any other replication scenerio within proxmox. Whether the VM or Container is a LAMP server or an NFS or SAMBA server isn't really all that different. A container "owns" a dataset as it has a bind mount to that local zfs filesystem (dataset), that same dataset exists on a sister storage node, at a previous snapshot level, just like any other replication scenario withing Proxmox. The Container serves that storage as CIFS or NFS to the cluster, which at least in the former works perfectly fine, and I assume so would the latter. The storage node fails, the container is migrated to another storage node via normal Proxmox HA. This provides for crash consistant continuity within the cluster. It's not perfect but it could achive RTOs and RPOs on par with a lot of storage arrays, and it isn't a big reach from where the product already is.
The first scenerio would be more than there is now but requires less to wrap your head around, it is just adding existing datasets to the datacenter list of replicatable objects, the 2nd scenerio seems like it would leverge a whole lot of what is already there and only really requires maybe an alternate way to give a container access to local zfs storage. The only caveat I see is a failure that happens before a snapshot send is complete, however that same scenerio exists inherently within any zfs send / recv scenario so I don't really see where it's any different than any other. Plus it would give you the ability to do snapshots and rollbacks as well.
The first solution has its drawbacks in that it wouldn't be clear which node has the most up to date copy as a replication source, but in the second solution the node the container is running on could be the designated master. All "regular" VMs and containers could simply mount as NFS or CIFS and always be on the same version of the data. This way some nodes could serve as network storage with HA serving their shares to other compute only nodes or even to each other.
While I can kind of understand why there are no provisions for the first scenerio, as you really don't know who has the authoritive copy, in the 2nd scenerio you do, or at least as much so as any other replication scenerio within proxmox. Whether the VM or Container is a LAMP server or an NFS or SAMBA server isn't really all that different. A container "owns" a dataset as it has a bind mount to that local zfs filesystem (dataset), that same dataset exists on a sister storage node, at a previous snapshot level, just like any other replication scenario withing Proxmox. The Container serves that storage as CIFS or NFS to the cluster, which at least in the former works perfectly fine, and I assume so would the latter. The storage node fails, the container is migrated to another storage node via normal Proxmox HA. This provides for crash consistant continuity within the cluster. It's not perfect but it could achive RTOs and RPOs on par with a lot of storage arrays, and it isn't a big reach from where the product already is.
The first scenerio would be more than there is now but requires less to wrap your head around, it is just adding existing datasets to the datacenter list of replicatable objects, the 2nd scenerio seems like it would leverge a whole lot of what is already there and only really requires maybe an alternate way to give a container access to local zfs storage. The only caveat I see is a failure that happens before a snapshot send is complete, however that same scenerio exists inherently within any zfs send / recv scenario so I don't really see where it's any different than any other. Plus it would give you the ability to do snapshots and rollbacks as well.