FOSS ZFS over iSCSI options for use with PM (2018)

FuriousGeorge

Active Member
Sep 25, 2012
78
2
28
I'm looking for something FOSS that would work as a storage appliance with a web based gui that can be used for SMB.

OMV appears to have ZFS iSCSI target support. Barring any currently unforeseen issues, this seems like it would make a good choice. The only downside is that it is not intended to scale beyond the SMB. Indeed, a contributor on the Proxmox wiki implies that this the case for all Linux-based solutions, GUI or no.

OminiOS or similar with napp-it does not appear to have that problem. The latter is not really FOSS, though it appears to have a crippled free version. The pros appear to be that it would fit my spec, aside from not being FOSS. The biggest con I can think of is that I don't really want to marry napp-it.

Am I missing any good ones?

In general, irrespective of the availability of a GUI, it seems like OpenSolaris and it's FOSS derivatives, for whatever reason, have the best reputation for use as a ZFS/iSCSI storage back-end. At least, that's the impression that I get based on what I've read.

Is that a fair statement?

Thanks for reading.
 

LnxBil

Famous Member
Feb 21, 2015
5,484
614
133
Germany
At the moment, you can only use ZFS-over-iSCSI with the OpenSolaris based stuff.

@mir is working (or was working, haven't seen any updates in the last couple of month on the dev-mailinglist) on a FreeNAS plugin for PVE to be able to use ZFS-over-iSCSI with FreeNAS. FreeNAS should be also a possibility for you, because it is free (BSD, not GPL)
 

mir

Famous Member
Apr 14, 2012
3,553
113
83
Copenhagen, Denmark
At the moment, you can only use ZFS-over-iSCSI with the OpenSolaris based stuff.

@mir is working (or was working, haven't seen any updates in the last couple of month on the dev-mailinglist) on a FreeNAS plugin for PVE to be able to use ZFS-over-iSCSI with FreeNAS. FreeNAS should be also a possibility for you, because it is free (BSD, not GPL)
I am still waiting for a code review from proxmox.
 

FuriousGeorge

Active Member
Sep 25, 2012
78
2
28
I have FreeNAS up and running w/ LVM iSCSI (I expected ZFS target to work at first). I read about the patch that exists for PM, and I'm pretty sure I found it on GIT, but I didn't check it out as it was a patch. It's good to hear that it may make its way into PM proper. So maybe I'll stick with FreeNAS for my GUI-based solution for now because of this.

Not that I mind playing with used-to-be-OpenSolaris for a CLI-only solution, in fact I look forward to it, but why is OMV not an option? I see reports of it working on these forums. Has something changed?
 

LnxBil

Famous Member
Feb 21, 2015
5,484
614
133
Germany
I cannot extract the information about ZFS-over-iSCSI in that post - read it 3 times. It is only said that he uses a zvol and iSCSI. He has not mentioned if he uses ZFS-over-iSCSI on the PVE side or just plain LVM, which works for every SAN implementation.

Just to be clear:
ZFS-over-iSCSI in PVE means that each VM has its own disk that is backed by a ZFS server over iSCSI and can be snapshotted, rollbacked and so on. Yet, you can also just use an iSCSI LUN on any PVE host with default Debian tools, create a physical volume and a volume group on top and use this in a clustered fashion for your VMs. With this setup, you do not have snapshot capability on your PVE host, just on the ZFS server, yet you do not have any benefits from both worlds with respect to PVE functionality.
 

FuriousGeorge

Active Member
Sep 25, 2012
78
2
28
I'm using LVM/iSCSI currently.

I just read a little too much into the statement:

"Now running 3.16 Kernel with a ZFS ZVol and iSCSI target."

I took the last part of the sentence, in the context of the ~4 year old note on the wiki, to be synonymous with ZFS over iSCSI

Your doubt makes it pretty clear this was not the intended meaning.

OmniOS or OpenIndiana and a CLI will be more fun anyway.

Also, having done a bit more research, it seems a Cephs cluster may make more sense for part of what I'm looking at.
 

LnxBil

Famous Member
Feb 21, 2015
5,484
614
133
Germany
Your doubt makes it pretty clear this was not the intended meaning.

I still hope I'm wrong, yet there was no big fuzz around it. It would be great to have other implementations for this.

I just looked at the code in /usr/share/perl5/PVE/Storage/ZFSPlugin.pm and it might just work. I'll try and be back.
 

LnxBil

Famous Member
Feb 21, 2015
5,484
614
133
Germany
So, I tried OpenMediaVault with istgt (other iscsi stuff is not working due to missing dependencies. iscsitarget is not in stretch anymore) and it does not work. Even after manually hacking in the PVE code. It seems that there is currently only NAS4Free and FreeNAS istgt support. The code is quite old as you can see in the GIT: https://git.proxmox.com/?p=pve-stor...4f77894db5e215f23ad4f8c57a721fdb4f46a;hb=HEAD

It could work yet I do not know to adapt the code without further debugging. I'll ask on the developer mailing list.
 

FuriousGeorge

Active Member
Sep 25, 2012
78
2
28
Thanks. You just saved me a bunch of time. It will be interesting to see how all of this evolves.

I think I'll leave FreeNAS in place for now and mess with the OS formerly known as OpenSolaris as an educational project.
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
6,199
1,056
164
I am still waiting for a code review from proxmox.

did I miss a new iteration after your v7? I sent reviews for that one back in August ;)
 

mir

Famous Member
Apr 14, 2012
3,553
113
83
Copenhagen, Denmark
did I miss a new iteration after your v7? I sent reviews for that one back in August ;)
Content of last email from you dated 2017-07-26:
sorry for the long delay / surprise absence. I had to start my vacation
a little bit earlier than planned, which meant not being able to finish
the current review round before that as originally intended.

just a short heads-up - I am back from my vacation, and will continue
reviewing your v7 in the next days, hopefully finishing at the start of
next week.
 

TBone

New Member
Jun 7, 2018
1
0
1
43
also intrested in freenas zfs over iscsi and news, is there a beta ?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!