Storage types and replication , NFS and local ZFS in a cluster

[FcT]

Hi Proxmox community,

I am currently running a 3 nodes cluster and all my VMs disks are stored on NFS shares provided by another physical server. As that NFS server is a SPOF, I would like to replicate the VM and their disks on my third node :
pve1 and pve2 have access to the nfs shares named SAS & SATA.
pve3 does not use the nfs shares, it has local disks and two ZFS pools named SAS & SATA (both created before joining the cluster), that will be used as a zfs replication target.

The gui (datacenter => storage => add => ZFS) let me browse to the local ZFS pools on pve3, but fails to create the storage with the same ID (SAS or SATA required by the replication, which makes sens). Is there a workaround (ZFS over iSCSI instead of NFS ?) to use theses local disks as an acceptable replication target ?

How would you improve the SPOF created by the nfs server, or how could I use my third server as a Business Continuity Plan ?

Thank you all,
FcT

 

[LnxBil]

it has local disks and two ZFS pools named SAS & SATA (both created before joining the cluster), that will be used as a zfs replication target.

replicated to what? You need a ZFS source and destination pool. Your NFS cannot be it, so you have only one pool on pve3.

How would you improve the SPOF created by the nfs server, or how could I use my third server as a Business Continuity Plan ?

I don't see a way with the hardware you have described. I would use a proper HA NFS solution or a SAN (dedicated as in a box with two controllers or distributed like ceph, starwind or drbd). You could also use ZFS replication inside of your cluster, but that is a PITA for me to setup and maintain and you would need the storage inside of the nodes, better to just go directly to a 3-node CEPH cluster, which is the minimum and would yield a real cluster where any component could fail without interrupting the overall PVE cluster.

 

[FcT]

Thank you for the clarification MnxBil, I wrongly thought that because the underlying file system of my nfs shares was ZFS, it might have been possible.
I now understand that I have nothing to replicate to pve3. I do come from the road you described : ZFS replication inside the cluster with local disks and pools in each server. It worked great for my environment, but was really a waste of disk space in the end.
Proper ha nfs or SAN is out of reach for my organisation, I will try to find a migration path to CEPH in a near future.

Thanks again for your insights and expertise !
FcT