FreeNAS 11.2u4 with Proxmox 5.4 iSCSI

[3degreesright]

Hello, my iSCSI target is a zvol on a traditional RAID and is NOT using ZFS on FreeNAS. I setup my first iSCSI target as a plain "iSCSI" storage. Should I consider using "ZFS over iSCSI" as a storage setup in this particular setup?

I have MPIO setup on FreeNAS, but have not yet setup MPIO on Proxmox. From what I gather, the best course of action is to first setup iSCSI, then enable MPIO on Proxmox using the following guide: https://pve.proxmox.com/wiki/ISCSI_Multipath

Any advice regarding the scenario above? Thank you.

 

[vshaulsk]

I run both MPIO ISCSI (Freenas ZVOL ---> 3 node proxmox cluster) and it works really good from a reliability and performance standpoint. I can saturate my 10gb links. I found the following forum post useful to setting it up https://forum.proxmox.com/threads/multipath-iscsi-lvm-and-cluster.11938/

I also run ZFS over ISCSI with the freenas patch. This was also easy to setup, but my performance is drastically worse than the standard ISCSI setup. I have not dug into why.......

 

[Quintin Cardinal]

I also run ZFS over ISCSI with the freenas patch. This was also easy to setup, but my performance is drastically worse than the standard ISCSI setup. I have not dug into why.......

That's interesting, I run the same ZFS over iSCSI patch in a completely virtual environment (FreeNAS, 2x PVE hosts, and an LXC container for corosync-qnetd running on their own network all virtualized on my single node PVE on ZFS physical host) and I didn't have any speed issues at all. A basic Windows VM running in that virtual lab environment was easily able to get 50+MB/s disk I/O.

Should I consider using "ZFS over iSCSI" as a storage setup in this particular setup?

I think the ZFS over iSCSI with FreeNAS works awesome. It's a lot of fun to work in.The only problem is that, because it's not natively available and a patch is required, PVE updates will break it and you won't have access to the FreeNAS storage until the patch is reapplied (and on each of your hosts, too.) I usually redownload the patch every time that happens, just in case there's changes, but either way it would be easy to script the process and make it a one command job to repatch! ZFS over iSCSI gives you some awesome functionality like each virtual disk is it's own thin provisioned (if you so choose) zvol, snapshots, and more!

 

[vshaulsk]

I like the features of ZFS over ISCSI in that it automatically creates ZVOL, which I can easily snapshot. However, again my performance is slower vs regular ISCSI.
Just quick example: Using proxmox host 2 (Dell R620) - Freenas 11.2-u4 (8x1tb enterprise SSD vdev mirrors); Freenas for ISCSI has its own dedicated multipath 10 gig network.

Running windows 10 VM and added two virtual disks. (Simple test showing difference in performance.... not accurate by the numbers...... however I can actually feel the dramatic difference in response and usability)
1) zfs over iscsi


2) using standard ISCSI multipath