Hi
I have a Windows 7 host with 1 x 32Gb virtual disk (IDE).
I have added a new 64Gb virtual disk (VIRTIO) and installed virtio drivers as required (Windows see the disk)
I have then booted the VM with a live CD of Ubuntu 11.10 and attempted to clone the IDE disk to the new VIRTIO disk using GPARTED for copying the partition. The process fails quite immediately reporting physical errors on the disk. Both virtual disks have been checked thoroughly and they have no problem.
I have also tried with Clonezilla... same problem.
I then reboot the VM normally from its main virtual disk and it wont boot anymore.
Proxmox log report:
Error: can't activate LV netdisk01:vm217-disk-1....
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 34363867136: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 34363924480: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 0: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 4096: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 64424443904: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 64424501248: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 0: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 4096: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 34363867136: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 34363924480: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 0: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 4096: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 64424443904: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 64424501248: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 0: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 4096: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 34363867136: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 34363924480: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 0: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 4096: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 64424443904: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 64424501248: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 0: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 4096: Input/output error
Volume group "vg01" not found
TASK ERROR: can't activate LV 'netdisk01:vm-217-disk-1': Skipping volume group vg01
I ssh into the node to check and maybe reactivate the volume manually but the physical and logical volumes have vanished in thin air!
Here is the output of pvscan and pvdisplay
root@nd02-cl01:~# pvscan
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 34363867136: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 34363924480: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 0: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 4096: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 64424443904: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 64424501248: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 0: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 4096: Input/output error
PV /dev/sdc VG vg02 lvm2 [2.94 TiB / 2.94 TiB free]
PV /dev/sda2 VG pve lvm2 [136.20 GiB / 4.00 GiB free]
Total: 2 [3.08 TiB] / in use: 2 [3.08 TiB] / in no VG: 0 [0 ]
root@nd02-cl01:~# pvdisplay
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 34363867136: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 34363924480: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 0: Input/output error
/dev/vg01/vm-217-disk-1: read failed after 0 of 4096 at 4096: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 64424443904: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 64424501248: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 0: Input/output error
/dev/vg01/vm-217-disk-2: read failed after 0 of 4096 at 4096: Input/output error
--- Physical volume ---
PV Name /dev/sdc
VG Name vg02
PV Size 2.94 TiB / not usable 2.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 771686
Free PE 771686
Allocated PE 0
PV UUID dPbdGc-UR64-dbvB-63Mi-eW8A-DMTI-Mi6IfU
--- Physical volume ---
PV Name /dev/sda2
VG Name pve
PV Size 136.20 GiB / not usable 3.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 34866
Free PE 1023
Allocated PE 33843
PV UUID cPVzOe-2U0O-QqNM-8JN3-XITV-kQGH-IWY2bH
and here is the output of the same commands after about 2 minutes (did nothing in between.... just waited)
root@nd02-cl01:~# pvscan
PV /dev/sdc VG vg02 lvm2 [2.94 TiB / 2.94 TiB free]
PV /dev/sdb VG vg01 lvm2 [2.94 TiB / 1.63 TiB free]
PV /dev/sda2 VG pve lvm2 [136.20 GiB / 4.00 GiB free]
Total: 3 [6.02 TiB] / in use: 3 [6.02 TiB] / in no VG: 0 [0 ]
root@nd02-cl01:~# pvdisplay
--- Physical volume ---
PV Name /dev/sdc
VG Name vg02
PV Size 2.94 TiB / not usable 2.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 771686
Free PE 771686
Allocated PE 0
PV UUID dPbdGc-UR64-dbvB-63Mi-eW8A-DMTI-Mi6IfU
--- Physical volume ---
PV Name /dev/sdb
VG Name vg01
PV Size 2.94 TiB / not usable 4.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 771660
Free PE 426056
Allocated PE 345604
PV UUID Muzs5x-FhAL-eWaQ-xntS-xO0Q-0fic-B4Xpob
--- Physical volume ---
PV Name /dev/sda2
VG Name pve
PV Size 136.20 GiB / not usable 3.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 34866
Free PE 1023
Allocated PE 33843
PV UUID cPVzOe-2U0O-QqNM-8JN3-XITV-kQGH-IWY2bH
All back to normal!
Of course, as a consequence, all VMs sitting on that storage will fail/crash.
The log of the SAN unit report nothing about the episode which leads me to believe that the problem may be on the Proxmox node.
root@nd02-cl01:~# pveversion -v
pve-manager: 2.0-33 (pve-manager/2.0/c598d9e1)
running kernel: 2.6.32-7-pve
proxmox-ve-2.6.32: 2.0-60
pve-kernel-2.6.32-6-pve: 2.6.32-55
pve-kernel-2.6.32-7-pve: 2.6.32-60
lvm2: 2.02.88-2pve1
clvm: 2.02.88-2pve1
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-1
pve-cluster: 1.0-23
qemu-server: 2.0-20
pve-firmware: 1.0-15
libpve-common-perl: 1.0-14
libpve-access-control: 1.0-16
libpve-storage-perl: 2.0-12
vncterm: 1.0-2
vzctl: 3.0.30-2pve1
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-4
ksm-control-daemon: 1.1-1
What may be causing this strange behaviour? Any hint would be greatly appreciated.
Thanks
