Fiber Chanel and Shared Storage - Snapshot supported (HA enabled)

[floh8]

I'm using it for years, so I would assume yes, but there is always ways to improve, so please share your view on this.


That's clear and the portal part, but it does not mean that you have bandwidth aggregation or even path failback on the initiator. Qemu is according to the documentation not able to login into multiple portals, just a single host, so ZFS-over-iSCSI is not able to use multiple portals like you would on a multipathed host. I don't see how that would work multipathed or even failback unless you have high available IP as the portal.

Did you somewhere read in the pve docu that pve uses the integrated iscsi tool of qemu?
qemu for blockdevices in pve is only used to look at the block device for dirty bitmaps and backup snapshot mode. How the block device is connected doesn't interrest qemu. U can simply try it out in a test environment if u not trust.
For your future know-how: it is possible to configure 2 or more ip address for a single portal. Look at the targetcli menu.

 

[LnxBil]

Did you somewhere read in the pve docu that pve uses the integrated iscsi tool of qemu?

The reference documentation does not mention neither multipath capability nor multipath not-capability of ZFS-over-iSCSI, so you have to inspect the qemu command line running the VM. The qemu process does indeed use direct iscsi connection, as already explained:

/usr/bin/kvm -id 101 -name vm101 [...] -drive file=iscsi://10.192.0.225/iqn.2003-01.org.linux-iscsi.pve.x8664:sn.870e7bb84d73/0,if=none,id=drive-scsi0,format=raw,cache=none,aio=io_uring,detect-zeroes=on -device scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,rotation_rate=1,bootindex=100

This is - as already explained - a SINGLE connection to the portal and it is not multipath-capable as outlined in the qemu documentation.
This is mentioned in the official wiki (last line), also mentioned in the forums a couple of times, e.g. here.

qemu for blockdevices in pve is only used to look at the block device for dirty bitmaps and backup snapshot mode. How the block device is connected doesn't interrest qemu.

The point was, that there is no block device, the iscsi initiator is directly in the qemu binary and does not have a block device in the linux kernel, because the kernel does not handle iscsi in this case, therefore no multipath. Check for yourself.

U can simply try it out in a test environment if u not trust.

Of course I don't trust random people on the internet especially if they tell me to trust them. Instant red flag.
As explained above, I tried it out and invalidated your point.

For your future know-how: it is possible to configure 2 or more ip address for a single portal. Look at the targetcli menu.

You still don't get my point. Just having many IP addresses does not magically get multipath. It's not that simple and qemu in the ZFS-over-iscsi case does NOT do it, even the connection setup does not allow it.

 

[VictorSTS]

Another proof about @LnxBil arguments is that you cannot use LXC container's disk on ZFS over iSCSI storage because on LXC there is no QEMU.

Maybe we are mixing terms and referring to different kinds of storage from PVE perspective even if they use the same technologies like ZFS and iSCSI?

 

[floh8]

So after testing zfs-over-iscsi one can confirm that pve zfs-over-iscsi implementation uses qemu integrated iscsi. So @LnxBil is right with his opinion. So no multipathing and RDMA with zfs-over-iscsi. Thats extrem disappointing.