After failed backup images disapeared

[raj]

Hi Team,

I have a cluster of 3 proxmox boxes and a nas linked to it.

Iscsi and lvn on that and all the images are/were there.

This morning I seen that ll the servers were unresponsive and that the backups were coming up with /dev/sdb errors.

I rebooted the whole nas, switched and servers and afterwards i could not boot any vm error could not activate the LV.

I did a psevm and the datastore where the images were/are has showing as zero and when i did a lvdisplay there was no images at all.

I tried to remove and mount the lvm again but now it shows as all the space is available and still no images.

Could anyone help pls.

Cheers,

Raj

 

[dietmar]

I did a psevm and the datastore where the images were/are has showing as zero

What exactly is 'psevm'?

 

[raj]

pvesm sorry

 

[raj]

so before removing the lvm, when I did a pvesm status, i got


/dev/sdb: Checksum error
/dev/sdb: Checksum error
Datastore lvm 0 0 0 0 100.00%

and if i tried to start a vm i was getting :

/dev/sdb: Checksum error
/dev/sdb: Checksum error
/dev/sdb: Checksum error
Volume group "Datastore" not found
TASK ERROR: can't activate LV 'Datastore:vm-100-disk-1': Skipping volume group Datastore


but now after removing and adding the lvm i get

Datastore lvm 1 1944076288 0 1944076288 0.50%

and when i try to start a vm i get :

TASK ERROR: can't activate LV 'Datastore:vm-100-disk-1': One or more specified logical volume(s) not found.

Is there anyway to recover from this

 

[raj]

If I try to do a restore, I get the following :

lvremove 'Datastore/vm-106-disk-1' error: One or more specified logical volume(s) not found. (500)

Cheers

 

[snowman66]

Can you see images on NAS or where were images stored?

What is the output of pvscan and vgscan?

ls -l /etc/lvm/backup

 

[raj]

I have actually just added the LUn and created the LVM again and did restores as there were about 20 boxes done.

Lets see if it happens again.

Cheers,

Raj