LV Thin Pool Failed - Power Outage

[Steve0]

Hello,
I wonder if someone could shed some light on this please.
I have a Dell r710 running a PERC 6i raid controller with proxmox 7.2 running on sda and a thin volume on sdb.
This was recently upgraded from 6.4 last Sunday.

Yesterday there was a power failure which has damaged my thin volume somehow. Proxmox boots ok, and I can access the WebUI.

In the syslog I see this

activating LV 'raid-thin/raid-thin' failed: Check of pool raid-thin/raid-thin failed (status:1). Manual repair required!

If i run thin_check I get this

root@r710:~# thin_check /dev/sdb
examining superblock
  superblock is corrupt
    bad checksum in superblock, wanted 2594599123

I've tried a repair, but it just exits and says Manual repair required. I can't seem to find any suggestion on how to perform a manual repair.

root@r710:~# lvconvert --repair raid-thin/raid-thin
  Child 334173 exited abnormally
  Repair of thin metadata volume of thin pool raid-thin/raid-thin failed (status:-1). Manual repair required!

I do have backups but I would really like to get this back online as the backups were a few days old at the failure.

Thanks in advance for any advice.

Cheers
Steve

 

[bbgeek17]

There are many articles on LVM repair on the net. I am not endorsing this one in particular https://mellowhost.com/billing/index.php?rp=/knowledgebase/65/How-to-Repair-a-lvm-thin-pool.html but it seems ok.

Keep in mind any command you are running now could be your last on this dataset. If the data is that valuable that you cant go back a few backups - duplicate your disk to a new one so you can try again if necessary.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Steve0]

For future reference, the response above failed to work. My guess is that it was a problem underneath the LV, I don't really have the expertise to diagnose it properly. In the end I tried to destroy the thin pool in the GUI. This was met with an error. So I had to wipe the disk. This worked and I was then able to re-create the pool and restore from my backups.

 

[koby]

I have the same problem , Any news about that recovery option ?

 

[Shannon Barber]

One drive with LVM corruptions means restore-from-backup.
Your hardware RAID controller has failed you.
Consider LVM-RAID w/JBOD - far more reliable.

Verify your drives support ZRAT/DRAT. If they do not then you should get new drives that do and failing that must disable TRIM and discard.