[SOLVED] Share ZFS Storage between Nodes

[Denny32]

Hi,

I am rather new to Proxmox and to an extent Linux.

I currently have a Cluster consisting of 2 Nodes. Both Servers have different sized Partitions. We use the larger Partition to store the VM's on. "zfs-pve01" and "zfs-pve02".
Now I want each host to have access to both zfs-storages to Migrate VM's seamlessly.
Both Storages are visible on each Host, but the "external" Storage is not active. In the Systemlog I get errors like:

Apr 09 13:35:22 proxmox1 pvestatd[1685]: zfs error: cannot open 'zfs-pve02': no such pool
Apr 09 13:35:22 proxmox1 pvedaemon[1707462]: zfs error: cannot open 'zfs-pve02': no such pool
Apr 09 13:35:22 proxmox1 pvedaemon[1707462]: could not activate storage 'zfs-pve02', zfs error: cannot import 'zfs-pve02': no such pool available
Apr 09 13:35:23 proxmox1 pvestatd[1685]: zfs error: cannot open 'zfs-pve02': no such pool

If i try to mount the Storage using something like:

mount proxmox2:/zfs-pve02 /mnt/zfs-pve02

I get errors like:

"Connection refused" or "unable to open datasheet"


Current configuration in the "/etc/pve/storage.cfg"

zfspool: zfs-pve02
pool zfs-pve02
content images,rootdir
mountpoint /zfs-pve02
nodes proxmox1,proxmox2
sparse 0

zfspool: zfs-pve01
pool zfs-pve01
content rootdir,images
mountpoint /zfs-pve01
nodes proxmox1,proxmox2
sparse 0

Does my plan even make sense? I heard, that zfs cannot be shared to other Servers in the Node. But it is apparently possible if both Storages have the same name. Since we have different sized Storages, that is also not a solution for us, since we would loose Storage space.


Thank you very much for any advice in advance.

If i need to provide any additional Information please let me know.

 

[Dunuin]

ZFS is a local and not a shared storage. You can`t remotely access it across nodes. Whats possible is to replicate virtual disks between two IDENTICALLY named ZFS pools so both pools store a local copy the same data (means also losing half the space as everything needs to be stored twice). But replicated virtual disks will never be perfectly synced. Minimum interval is 1 minute, so in case a node goes down you lose up to 1 minute of data.

For a real shared storage without SPOF you should have a look at ceph. But this requires proper networking and minimum of 3 nodes (better 5 or more).

 

[bbgeek17]

I heard, that zfs cannot be shared to other Servers in the Node.

Thats correct. ZFS is a local filesystem that does not have built-in functionality to be "shared". You can use other add-ons to expose the context of the filesystem to the network, i.e. NFS.

But it is apparently possible if both Storages have the same name

Thats not "sharing". Its a trick that allows Proxmox to do a full data copy when you want to migrate. To avoid having to specify target storage name, Proxmox simply expects a pool with the same name to exist. However, as noted above - thats not "shared", just same-named.

Since we have different sized Storages, that is also not a solution for us, since we would loose Storage space.

The size does not matter, as long as there is enough for a full copy of your data.
But, as mentioned previously, it only facilitates "migration with local storage" which means full copy every time.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox