the issue appear when migrating vm disk from nfs to lvm on top iscsi. then we test to manually create lv from cli
Status of the multipath
vgs result, there is still free remaining
We used qsan storage with iscsi protocol and thin pool lun. and checked on the qsan dashboard there are no issued regarding space or locked lun.
the logs on systemd
Code:
root@pve02:~# lvcreate -L 100G -n test_vol Storage-QSAN
Error reading device /dev/mapper/mpathqsan at 26388279001088 length 4.
Error reading device /dev/mapper/mpathqsan at 26388279058432 length 4.
Error reading device /dev/mapper/mpathqsan at 26388279065600 length 24.
Error reading device /dev/mapper/mpathqsan at 26388279066112 length 512.
Error writing device value /dev/Storage-QSAN/test_vol at 0 length 4096.
Failed to initialize logical volume Storage-QSAN/test_vol with value 0 and size 4.00 KiB.
Aborting. Failed to wipe start of new LV.
Code:
copying volume 'StandardStorage:209/vm-209-disk-1.qcow2' from current storage 'StandardStorage' to target storage 'Standart-QSAN'
create full clone of drive scsi1 (StandardStorage:209/vm-209-disk-1.qcow2)
Rounding up size to full physical extent 50.01 GiB
Logical volume "vm-209-disk-1.qcow2" created.
Formatting '/dev/Storage-QSAN/vm-209-disk-1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=53687091200 lazy_refcounts=off refcount_bits=16
qemu-img: Failed to flush the L2 table cache: Input/output error
qemu-img: Failed to flush the refcount block cache: Input/output error
Logical volume "vm-209-disk-1.qcow2" successfully removed.
TASK ERROR: storage migration failed: unable to create image: qemu-img: /dev/Storage-QSAN/vm-209-disk-1.qcow2: Could not resize image: Allocating clusters failed: Input/output error
Status of the multipath
Code:
root@pve02:~# multipath -ll
mpathqsan (3201a0013780ffa00) dm-54 Qsan,XS3212
size=24T features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=50 status=active
|- 10:0:0:0 sdf 8:80 active ready running
`- 9:0:0:0 sde 8:64 active ready running
vgs result, there is still free remaining
Code:
root@pve02:~# vgs
Error reading device /dev/mapper/mpathqsan at 26388279001088 length 4.
Error reading device /dev/mapper/mpathqsan at 26388279058432 length 4.
Error reading device /dev/mapper/mpathqsan at 26388279065600 length 24.
Error reading device /dev/mapper/mpathqsan at 26388279066112 length 512.
VG #PV #LV #SN Attr VSize VFree
Storage-QSAN 1 60 0 wz--n- <24.00t <21.22t
ceph-2757eb11-5a6d-476d-97e8-e59fd8de01ae 1 1 0 wz--n- <4.55t 0
pve 1 2 0 wz--n- <930.00g 0
We used qsan storage with iscsi protocol and thin pool lun. and checked on the qsan dashboard there are no issued regarding space or locked lun.
the logs on systemd
Code:
Sep 15 15:04:48 pve01 kernel: sd 9:0:0:0: alua: supports implicit TPGS
Sep 15 15:04:48 pve01 kernel: sd 9:0:0:0: alua: No device descriptors found
Sep 15 15:04:48 pve01 kernel: sd 9:0:0:0: alua: port group 0 rel port 1
Sep 15 15:04:48 pve01 kernel: sd 9:0:0:0: [sde] Very big device. Trying to use READ CAPACITY(16).
Sep 15 15:04:48 pve01 kernel: sd 9:0:0:0: alua: transition timeout set to 60 seconds
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#69 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#69 Sense Key : Data Protect [current]
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#69 Add. Sense: Space allocation failed write protect
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#69 CDB: Read(16) 88 00 00 00 00 0b ff ff ff 00 00 00 01 00 00 00
Sep 15 15:04:49 pve01 kernel: critical space allocation error, dev sdd, sector 51539607296 op 0x0:(READ) flags 0x4200 phys_seg 32 prio class 2
Sep 15 15:04:49 pve01 kernel: critical space allocation error, dev dm-23, sector 51539607296 op 0x0:(READ) flags 0x0 phys_seg 32 prio class 2
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#63 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#63 Sense Key : Data Protect [current]
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#63 Add. Sense: Space allocation failed write protect
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#63 CDB: Read(16) 88 00 00 00 00 0b ff ff ff 00 00 00 01 00 00 00
Sep 15 15:04:49 pve01 kernel: critical space allocation error, dev sde, sector 51539607296 op 0x0:(READ) flags 0x4200 phys_seg 32 prio class 2
Sep 15 15:04:49 pve01 kernel: critical space allocation error, dev dm-23, sector 51539607296 op 0x0:(READ) flags 0x0 phys_seg 32 prio class 2
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#26 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#26 Sense Key : Data Protect [current]
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#26 Add. Sense: Space allocation failed write protect
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#26 CDB: Read(16) 88 00 00 00 00 0b ff ff ff 00 00 00 01 00 00 00
Sep 15 15:04:49 pve01 kernel: critical space allocation error, dev sdd, sector 51539607296 op 0x0:(READ) flags 0x4200 phys_seg 32 prio class 2
Sep 15 15:04:49 pve01 kernel: critical space allocation error, dev dm-23, sector 51539607296 op 0x0:(READ) flags 0x0 phys_seg 32 prio class 2
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#110 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#110 Sense Key : Data Protect [current]
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#110 Add. Sense: Space allocation failed write protect
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#110 CDB: Read(16) 88 00 00 00 00 0b ff ff ff 00 00 00 01 00 00 00
Sep 15 15:04:49 pve01 kernel: critical space allocation error, dev sde, sector 51539607296 op 0x0:(READ) flags 0x4200 phys_seg 32 prio class 2
Sep 15 15:04:49 pve01 kernel: critical space allocation error, dev dm-23, sector 51539607296 op 0x0:(READ) flags 0x0 phys_seg 32 prio class 2
Sep 15 15:04:49 pve01 pveproxy[2501214]: got inotify poll request in wrong process - disabling inotify
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#2 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#2 Sense Key : Data Protect [current]
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#2 Add. Sense: Space allocation failed write protect
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#2 CDB: Read(16) 88 00 00 00 00 0b ff ff ff 00 00 00 01 00 00 00
Sep 15 15:04:49 pve01 kernel: critical space allocation error, dev sdd, sector 51539607296 op 0x0:(READ) flags 0x4200 phys_seg 32 prio class 2
Sep 15 15:04:49 pve01 kernel: critical space allocation error, dev dm-23, sector 51539607296 op 0x0:(READ) flags 0x0 phys_seg 32 prio class 2
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#77 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#77 Sense Key : Data Protect [current]
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#77 Add. Sense: Space allocation failed write protect
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#77 CDB: Read(16) 88 00 00 00 00 0b ff ff ff 00 00 00 01 00 00 00
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#3 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#3 Sense Key : Data Protect [current]
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#3 Add. Sense: Space allocation failed write protect
Sep 15 15:04:49 pve01 kernel: sd 8:0:0:0: [sdd] tag#3 CDB: Read(16) 88 00 00 00 00 0b ff ff ff 00 00 00 01 00 00 00
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#78 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#78 Sense Key : Data Protect [current]
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#78 Add. Sense: Space allocation failed write protect
Sep 15 15:04:49 pve01 kernel: sd 9:0:0:0: [sde] tag#78 CDB: Read(16) 88 00 00 00 00 0b ff ff ff 00 00 00 01 00 00 00

