ZFS on iSCSI with HA/Redundancy via LACP

[EagleTG]

Hi all,

This is sort of a follow-up from my post regarding Corosync, but different enough that I thought I'd post a new thread, and bring it to the Configuration forum. I'm familiar with the VMware-style of configuration of iSCSI in an HA scenario utilizing MPIO/multipathing and am interested in building a similar environment with Proxmox. I have been using FreeNAS as my storage back-end in my various VMware builds, and am continuing to do so with the Proxmox build.

I've considered the drawbacks of a true multipath iSCSI configuration utilizing LVM to the back end storage but really would like to keep Proxmox benefits like snapshotting available in the environment. I've also considered the performance benefits multipathing can provide, and I'm OK with how things are currently performing over a single 10Gbe link throughout my testing thus far. Therefore, I've decided on ZFS over iSCSI and have installed the GrandWazoo plugin. Things are working great, and speeds are perfect. This naturally brings me to the question of how to properly configure this with some form of HA while not introducing any potential stability concerns that may be brought on by running iSCSI traffic over LACP (out of order frame delivery and other concerns). Naturally, thinking of sticking to 802.3ad rather than round-robin for just the out-of-order frame concern.

This FreeNAS server is dedicated to the Proxmox cluster I'm building, so there is no concern over creating an LACP bond with the physical FreeNAS ports.

I've read a bunch of the information on the forums and combined with prior experience really am hesitant to run iSCSI over an LACP bond. Unfortunately, I don't see any other way to make this happen. Here's my proposed configuration:

​

The grey IP Addresses are currently configured while I messed around with non-ZFS iSCSI multipathing and would be dropped with the potential move to an LACP/Port-Channel configuration. Naturally VLAN 121 would also be dropped.

TLDR; I basically wanted to see if those who are using ZFS over iSCSI have taken the same approach of using LACP for their HA/Redundancy needs? If not, what did you decide to use?

Thoughts?

Thanks in advance!!

Edit: And I completely acknowledge that the FreeNAS is not redundant in this configuration. The goal is network redundancy.

 

[guletz]

Hi,

Maybe ospf ? Fail-over and load balancing over this 2 iscsi links.

Good luck / Bafta

 

[LnxBil]

TLDR; I basically wanted to see if those who are using ZFS over iSCSI have taken the same approach of using LACP for their HA/Redundancy needs? If not, what did you decide to use?

You mean the ZFS-over-iSCSI that is ZFS on the server and using it via QEMU directly? There has been some discussion about this and IIRC, there is or was no multipathing support on the KVM/QEMU side of things, therefore only one link was used. The connection is directly done in KVM/QEMU, so there is no OS multipathing involved.

 

[EagleTG]

You mean the ZFS-over-iSCSI that is ZFS on the server and using it via QEMU directly? There has been some discussion about this and IIRC, there is or was no multipathing support on the KVM/QEMU side of things, therefore only one link was used. The connection is directly done in KVM/QEMU, so there is no OS multipathing involved.

Hi LnxBil,

Yes, I have seen some of that discussion as well so I'm looking for what work-arounds others have implemented to achieve some level of HA for their iSCSI connections.

I will research guletz' recommendation of OSPF in addition to testing LACP.

Thanks!