ZFS over ISCSI ?

[Roggar]

Hi,
I want to interconnect PVE 5.2-8 with a ZFS on FreeBSD 9.2 (yes this system is lock on this version..) for storage of VMs disk.
What is your setup recommendation for production ?
- ZFS over ISCSI ?
- One ZVOL on ZFS and ISCSI -> Proxmox LVM-Thin, LVM or other FS ?
- One ZFS dataset, NFS share with Proxmox -> qcow2 ?

I'm interested to give a try with ZFS over ISCSI, but it seems Proxmox send "istgt onerestart" commands to the FreeBSD Storage when you restore a Proxmox snapshot, I'm afraid that close all iscsi session and lock VMs. Have you any experience feedback for this point ?

I'm also interested by Proxmox VMs live snapshot, so in you opinion, what is the best setup for production ?

Thank you for your help !

 

[Roggar]

Hi,
Does anyone run on ZFS over iSCSI ?

 

[np86]

Hi,
Does anyone run on ZFS over iSCSI ?

I tested it some time ago. Decided to drop it since apparently LXC is not supported on that.
FreeNAS is not supported anymore, so I went with Nas4Free for testing if I remember correct.

I really wish there were some shared storage with the same features as ZFS (Local)

 

[raku]

I run ZFS over iSCSI on FreeNAS 11.1 using these patches: https://github.com/TheGrandWazoo/freenas-proxmox
So far it works like a charm. But as @np86 said - no LXC, KVM VMs only.

 

[np86]

I run ZFS over iSCSI on FreeNAS 11.1 using these patches: https://github.com/TheGrandWazoo/freenas-proxmox
So far it works like a charm. But as @np86 said - no LXC, KVM VMs only.

As far as I know, if we want shared storage with snapshot etc. CEPH is the only option and that is just too advanced for my small homelab setup

 

[raku]

ZFS over iSCSI supports snapshots. But as I said before - you can run KVM VMs only.

 

[nomad]

Hi, there is som fork of the git you point to this one seems to be active :
github.com/UstyuzhaninAV/freenas-proxmox

 

[raku]

@nomad - https://github.com/UstyuzhaninAV/freenas-proxmox/commits/master
Nothing has changed comparing to the source repo.

EDIT: it's even 2 commits behind the original repo.

 

[nomad]

You are right.
I don't try it yet.
does it permit to manage totaly zfs over iSCSI storage ?

 

[raku]

What do you mean by totaly?
These patches allow you to use FreeNAS as a iSCSI storage for your VM's disks. Proxmox can create/destroy/resize virtual disks which are iSCSI LUNs on FreeNAS.

 

[nomad]

I think you answer my question.

In fact I trieded several month ago to manage a zfs over iscsi storage (without the patch).
This storage was a zfs on a FreeBSD 11, so native iscsi.
And each time I wanted to manage my storage from my Proxmox page I have in first to create the zfs vol, then to edit by hand /etc/ctl.conf and reload ctld service on my FreeBSD server.

How does this patch manage all of this ?

 

[raku]

After applying patches, you can configure access to FreeNAS API via Proxmox GUI. Patches also added support for Proxmox to be able to manage LUN and ZFS configuration via API calls.

 

[Norman Uittenbogaart]

I'm wondering is there a performance benefit to running iSCSI instead of NFS to FreeNAS?
Especially as the ZFS over iSCSI does not support MPIO yet according to the Wiki?
(I would bundle 2 x 1Gbit connections)

 

[HybridSarcasm]

I'm wondering is there a performance benefit to running iSCSI instead of NFS to FreeNAS?
Especially as the ZFS over iSCSI does not support MPIO yet according to the Wiki?
(I would bundle 2 x 1Gbit connections)

Not sure about performance impacts, but the availability of snapshots over iSCSI is a huge one for me.

 

[Norman Uittenbogaart]

Not sure about performance impacts, but the availability of snapshots over iSCSI is a huge one for me.

Although trying to roll back snapshots made by FreeNAS can't be rolled back unless you disconnect the iSCSI target.