ZFS over iSCSI on Synology

[Carsten Czichos]

Hi there,
I have a Synology with current DSM 6.2.2.

I want to use ZFS over iSCSI because I need snapshots. Has anybody done that yet?
Is it possible to use LVM thin If I want to use the volume only on one Cluster-Node at a time?

Any help would be greatly appreciated.


Thanks, Carsten

 

[wolfgang]

Hi,

ZFS over iSCSI needs on the NAS side ZFS.
Synology uses BTRFS, so this does not work.

 

[Carsten Czichos]

Hi Wolfgang,

thank your for your quick reply !
I found on the roadmap that, that there are plans to support btrfs - but they are postponed. Any idea when that might happen?

Second question: can I mount iSCSI localy and then use lvm-thin on that directory? What would be the best approach?

Thanks again !

 

[wolfgang]

I found on the roadmap that, that there are plans to support btrfs - but they are postponed. Any idea when that might happen?

Yes, for local BTRFS but not remote.
If you want snapshots with Synology use CIFS/NFS.

Second question: can I mount iSCSI localy and then use lvm-thin on that directory? What would be the best approach?

As long you use it as exclusive non shared LUN, the answer is YES.
But if you mount the LUN on another node at the same time the Thin-LVM will be instant corrupt.

 

[Carsten Czichos]

Hi Wolfgang,
created CIFS and NFS-Storage with the Proxmox-Gui, but I can't take snapshots - see attached screeshot.
What am I missing ?

Thanks, Carsten

 

[mir]

The disk format must be Qcow2 to be able to take snapshots and you have probably chosen raw.

 

[Carsten Czichos]

actually I want containers and not kvm

 

[Carsten Czichos]

I have the Idea, that containers have less overhead in general and especially for backup and restore.

What is your opinion ?

 

[cpzengel]

I am using it for backup local zfs
Working for years
I can post directions if you like

 

[Carsten Czichos]

That would be great, thanks !

 

[cpzengel]

Basicly a few Steps

On old NAS create iSCSI Share for the Space you want to use with ZFS
Create a Datastore Type iSCSI in Proxmox VE to the one just created and disable it for usage as a Datastore
On Terminal find the created Disk under /dev/disk/by-id
Create zpool "zpool create MyNASZFS /dev/disk/by-id/wwn---- or similar (optional ashift=12, compression, etc)
Create a Dataset Type ZFS found as MyNASZFS or any Name you given

Please be aware that it´s not recommendet, but working for me as reliable Solution for Replicas for Years

Summary, you connect to an iSCSI Storage via Ethernet. The virtual Disk will be formated as a ZVOL and used as a Datastore or Replication Target

 

[Carsten Czichos]

Thanks, I will try that !

 

[Witker]

Was It Successfully?
Or any news Dec.2020 for other solution?

 

[booradlly]

Basicly a few Steps

On old NAS create iSCSI Share for the Space you want to use with ZFS
Create a Datastore Type iSCSI in Proxmox VE to the one just created and disable it for usage as a Datastore
On Terminal find the created Disk under /dev/disk/by-id
Create zpool "zpool create MyNASZFS /dev/disk/by-id/wwn---- or similar (optional ashift=12, compression, etc)
Create a Dataset Type ZFS found as MyNASZFS or any Name you given

Please be aware that it´s not recommendet, but working for me as reliable Solution for Replicas for Years

Summary, you connect to an iSCSI Storage via Ethernet. The virtual Disk will be formated as a ZVOL and used as a Datastore or Replication Target

Am I correct that this will not work on Synology ?

 

[bbgeek17]

I dont see why the procedure described wont work with any iSCSI capable server. It's a bit hacky, i.e. creating iSCSI proxmox datastore just to get disk attached.. I'd recommend using iscsiadm. Other than that, what is described here is connecting iscsi disk and formatting it with a volume manager. It has no dependency on either PVE or Synology...

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[booradlly]

Yes, for local BTRFS but not remote.
If you want snapshots with Synology use CIFS/NFS.

Would you agree with the following edit ?
"If you want snapshots with Synology use CIFS/NFS, with qcow (vmdk ?) not raw."

As long you use it as exclusive non shared LUN, the answer is YES.
But if you mount the LUN on another node at the same time the Thin-LVM will be instant corrupt.

Based on this comment shouldn't I be able use lvm-thin on Synology ?
Synology iSCSI -> lvm -> lvm-thin -> raw snapshots ?
Obviously this doesnt work in the UI, just wondering if its just not implemented, or if it wont work technically.

 

[bbgeek17]

this whole thread is old and a few completely different questions are discussed here. My prior comment was only regarding what you quoted, which in fact had nothing to do with this thread's topic.

"If you want snapshots with Synology use CIFS/NFS, with qcow (vmdk ?) not raw."

Its partially correct. For a novice user the qcow on CIFS/NFS is the easiest approach. A more advanced user could manually attach iscsi LUN, create thin LVM on it and utilize snapshots. It just CANNOT be used as shared storage.

Based on this comment shouldn't I be able use lvm-thin on Synology ?
Synology iSCSI -> lvm -> lvm-thin -> raw snapshots ?

yes when done manually

Obviously this doesnt work in the UI, just wondering if its just not implemented, or if it wont work technically.

because too many layers get involved and maintenance becomes complicated. If you want LVM thin - deal with your disks (iscsi,FC,sas,nvmeof) on your own.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[michael_hess]

Basicly a few Steps

On old NAS create iSCSI Share for the Space you want to use with ZFS
Create a Datastore Type iSCSI in Proxmox VE to the one just created and disable it for usage as a Datastore
On Terminal find the created Disk under /dev/disk/by-id
Create zpool "zpool create MyNASZFS /dev/disk/by-id/wwn---- or similar (optional ashift=12, compression, etc)
Create a Dataset Type ZFS found as MyNASZFS or any Name you given

Please be aware that it´s not recommendet, but working for me as reliable Solution for Replicas for Years

Summary, you connect to an iSCSI Storage via Ethernet. The virtual Disk will be formated as a ZVOL and used as a Datastore or Replication Target

I'm new to ProxMox, got an old Rackstation, can get iSCSI and LVM working, but can't get ZFS on the LVM. Not sure that's what you mean, but I can't follow your steps.

I connected iSCSI to three LUN's, can see and create on it fine. Nothing in by-id showed up when adding iscsi. Added LVM to the first LUN, can see that, so tried:

lvm-pv-uuid-PevIB4-oLJ9-4RdD-lhOZ-LbQG-SMIE-panjYJ
...
wwn-0x600508b1001cc4789a704e7c1dfb2e32
wwn-0x600508b1001cc4789a704e7c1dfb2e32-part1
wwn-0x600508b1001cc4789a704e7c1dfb2e32-part2
wwn-0x600508b1001cc4789a704e7c1dfb2e32-part3
root@proxmox-srv0-n1:~# zpool create RS-SSD-ZFS /dev/disk/by-id/lvm-pv-uuid-COYxeL-3QvW-x6zy-GHa5-QMy3-0RUt-q78v7S
invalid vdev specification
use '-f' to override the following errors:
/dev/disk/by-id/lvm-pv-uuid-COYxeL-3QvW-x6zy-GHa5-QMy3-0RUt-q78v7S contains a filesystem of type 'LVM2_member'
root@proxmox-srv0-n1:~# zpool create RS-SSD-ZFS /dev/disk/by-id/lvm-pv-uuid-COYxeL-3QvW-x6zy-GHa5-QMy3-0RUt-q78v7S -f
missing link: sdb was partitioned but /dev/disk/by-id/lvm-pv-uuid-COYxeL-3QvW-x6zy-GHa5-QMy3-0RUt-q78v7S-part1 is missing
root@proxmox-srv0-n1:~#

I need snaps basically, and can't get lvm-thin working as I have 3 hosts that need to share the lun's. Any thoughts would be appreciated!

 

[bbgeek17]

can get iSCSI and LVM working, but can't get ZFS on the LVM

You are confusing combinations of technologies:
Non-shared iSCSI + ZFS = possible. not recommended
Shared iSCSI + LVM = possible
ZFS internally on storage devices exported via iSCSI = possible in some cases

Shared iSCSI + ZFS = NOT possible

I think you misunderstood the ZFS over iSCSI scheme. Its a special mode where a particular storage device:
a) allows management via SSH/root
b) uses ZFS internally
c) is able to export above ZFS via iSCSI
d) compatible with available ZFS/iSCSI plugin

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[michael_hess]

You are confusing combinations of technologies:
Non-shared iSCSI + ZFS = possible. not recommended
Shared iSCSI + LVM = possible
ZFS internally on storage devices exported via iSCSI = possible in some cases

Shared iSCSI + ZFS = NOT possible

I think you misunderstood the ZFS over iSCSI scheme. Its a special mode where a particular storage device:
a) allows management via SSH/root
b) uses ZFS internally
c) is able to export above ZFS via iSCSI
d) compatible with available ZFS/iSCSI plugin

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

Thank you for clarifying that.

What would you suggest is the best combination to imitate (or improve on) VMWare? With Multipath, iSCSI, capable storage arrays? I've got a Racksation, Dell Powershare, Dell PowerStore, and an HP 3PAR. All VMFS via iSCSI, shared between 7 hosts.

Shared iSCSI + LVM would be great but I need snapshot support for our devs.

Appreciate the help!