Cannot delete snapshot and disk move fails on possible corrupt LV

[zhoid]

Good Day,

I started noticing the following alerts when moving disks between LUNS in my environment


WARNING: /dev/PVE03/vm-137-disk-0: Thin's thin-pool needs inspection.
WARNING: /dev/PVE03/vm-156-disk-0: Thin's thin-pool needs inspection.
WARNING: /dev/PVE03/vm-157-disk-0: Thin's thin-pool needs inspection.
/dev/coraid-ssd-2/vm-135-disk-0: read failed after 0 of 4096 at 0: Input/output error
/dev/coraid-ssd-2/vm-135-disk-0: read failed after 0 of 4096 at 214748299264: Input/output error
/dev/coraid-ssd-2/vm-135-disk-0: read failed after 0 of 4096 at 214748356608: Input/output error
/dev/coraid-ssd-2/vm-135-disk-0: read failed after 0 of 4096 at 4096: Input/output error
WARNING: /dev/PVE03/vm-158-disk-0: Thin's thin-pool needs inspection.
WARNING: /dev/PVE03/vm-158-disk-1: Thin's thin-pool needs inspection.

After running lvs -a PVE03

LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
PVE03 PVE03 twi-cotzM- 3.64t 94.95 48.96
[PVE03_tdata] PVE03 Twi-ao---- 3.64t
[PVE03_tmeta] PVE03 ewi-ao---- 120.00m
[lvol0_pmspare] PVE03 ewi-a----- 120.00m
snap_vm-137-disk-0_upgrade PVE03 Vri---tz-k 700.00g PVE03 vm-137-disk-0
vm-137-disk-0 PVE03 Vwi-aotz-- 700.00g PVE03 85.39
vm-154-disk-0 PVE03 Vwi---tz-- 700.00g PVE03
vm-156-disk-0 PVE03 Vwi-aotz-- 850.00g PVE03 77.37
vm-157-disk-0 PVE03 Vwi-aotz-- 950.00g PVE03 96.60
vm-158-disk-0 PVE03 Vwi-aotz-- 950.00g PVE03 85.28
vm-158-disk-1 PVE03 Vwi-a-tz-- 50.00g PVE03 38.87

I noticed a snapshot I could not receive from the GUI OR CLI

I started moving disks off PVE03 to another LUN as my intention is to destroy PVE03, check the disks and rebuild it if the disks are fine, these issues could have occurred as PVE03 Thin LV ran out of space recently after taking snapshots and backups were made.

I tried moving disk of VM 157 ( vm-157-disk-0 ), I leave the checkbox untick to delete the source intentionally though..

After 3 hours the dis move completes but fails with the below error.

I have tried this 3 times now, the same result.

WARNING: /dev/PVE03/vm-137-disk-0: Thin's thin-pool needs inspection.
WARNING: /dev/PVE30/vm-157-disk-0: Thin's thin-pool needs inspection.
WARNING: /dev/PVE03/vm-156-disk-0: Thin's thin-pool needs inspection.
WARNING: /dev/PVE03/vm-157-disk-0: Thin's thin-pool needs inspection.
/dev/coraid-ssd-2/vm-135-disk-0: read failed after 0 of 4096 at 0: Input/output error
/dev/coraid-ssd-2/vm-135-disk-0: read failed after 0 of 4096 at 214748299264: Input/output error
/dev/coraid-ssd-2/vm-135-disk-0: read failed after 0 of 4096 at 214748356608: Input/output error
/dev/coraid-ssd-2/vm-135-disk-0: read failed after 0 of 4096 at 4096: Input/output error
WARNING: /dev/PVE03/vm-158-disk-0: Thin's thin-pool needs inspection.
Logical volume "vm-157-disk-0" successfully removed
WARNING: /dev/PVE03/vm-158-disk-1: Thin's thin-pool needs inspection.
TASK ERROR: storage migration failed: mirroring error: drive-scsi0: mirroring has been cancelled


Please advise how I can fix the issue OR underlying issue on PVE03 OR move the disks in another way as I don't mind destroying PVE03 and rebuilding it.

Thanks

Zaid

 

[zhoid]

Hi,

I have managed to migrate most of the VM's off from my LUN PVE03, I cannot migrate one of my disks.

How do I fix/repair this issue

WARNING: /dev/PVE03/vm-137-disk-0: Thin's thin-pool needs inspection.
WARNING: /dev/PVE30/vm-157-disk-0: Thin's thin-pool needs inspection.
WARNING: /dev/PVE03/vm-156-disk-0: Thin's thin-pool needs inspection.
WARNING: /dev/PVE03/vm-157-disk-0: Thin's thin-pool needs inspection.

Regards

Zaid

 

[zhoid]

After reading some forums a repair needs to be run on the LV

I have deactivated the LV's on PVE03
Deactivated PVE03/PVE03
Ran the following command

root@pve-205:~# lvconvert --repair PVE03/PVE03

it's on the following line for the last 12 hours

Using default stripesize 64.00 KiB.


I am not sure if anything is happening?

PVE03 is a 3.64Tb SSD LUN

Regards

Zaid