ZFS over ISCSI lun mapping issue

[mwarchut]

I am trying to migrate a bunch of VMs to a new ZFS over ISCSI box. I have some moved over but suddenly it stopped taking migration.s

when I attempt the migration I get this error.

TASK ERROR: storage migration failed: copy failed: command '/usr/bin/qemu-img convert -p -n -f raw --target-image-opts /dev/hpstorage/vm-159-disk-0 'file.driver=iscsi,file.transport=tcp,file.initiator-name=iqn.1993-08.org.debian:01:3c9d3216744f,file.portal=172.16.20.50,file.target=iqn.2003-01.org.linux-iscsi.rw-zfs-01.x8664:sn.7382e37c0378,file.lun=15,driver=raw'' failed: exit code 1

It appears to be attempting to access lun15. However there is no LUN15.

iscsi-inq iscsi://m%m@172.16.20.50:3260/iqn.2003-01.org.linux-iscsi.rw-zfs-01.x8664:sn.7382e37c0378/15
Login Failed. SENSE KEY:ILLEGAL_REQUEST(5) ASCQ:LOGICAL_UNIT_NOT_SUPPORTED(0x2500)

However the disk in question is at a different lun.

Peripheral Qualifier:CONNECTED
Peripheral Device TypeIRECT_ACCESS
Removable:0
Version:6 unknown
NormACA:0
HiSup:0
ReponseDataFormat:2
SCCS:1
ACC:0
TPGS:3
3PC:1
Protect:0
EncServ:0
MultiP:0
SYNC:0
CmdQue:1
Vendor:LIO-ORG
Product:stg-vm-159-disk
Revision:4.0
Version Descriptor:00a0 unknown
Version Descriptor:0960 iSCSI
Version Descriptor:0460 SPC-4
Version Descriptor:04c0 SBC-3


Any idea why it is doing this or how to deal with it?

TIA.

 

[tcabernoch]

Ok. I was going to make my own post looking for iSCSI recommendations.
But here's your post, and you are not following the published recommendations.
So that's a really good place to start.

Here is how the vendor tells us to set up iSCSI.
https://pve.proxmox.com/wiki/Multipath#LVM_on_top_of_a_LUN

You are not doing that. You are using ZFS.
Why? Is that the latest thing?
I like ZFS, but I like things to work. I did LVM-on-LUN, just like they said.
Mine works.

So ... at the bottom of that page is a section about ""Device mismatch detected" warnings" and other troubleshooting info.
Take a look.

 

[UdoB]

Why? Is that the latest thing?

It is this: https://pve.proxmox.com/wiki/Storage:_ZFS_over_ISCSI

iSCSI is "only" the transport for a ZVOL. All(?) ZFS related commands and features are available, but are executed on the target.

This sounds great and I had tested it once, some years ago. Unfortunately I dropped that approach and so I have no experience to help the OP.

 

[tcabernoch]

Ooooh. Ok. I just read all that.

This backend accesses a remote machine having a ZFS pool as storage and an iSCSI target implementation via ssh. For each guest disk it creates a ZVOL and, exports it as iSCSI LUN. This LUN is used by Proxmox VE for the guest disk.

This sounds like a setup I tested with a PBS accessing a zvol on TrueNas via iSCSI. Except there's a plugin that lets the PVE server manage the situation? Strangely that page mentions the plugin but provides no detail. Ya, I'm never gonna build this either. LVM is stable and simple.