Hi all,
This has happened to me 3 times in the past week and has never been a problem before. Am a little baffled as to what is causing it.
Basically the scheduled vzdump snapshot starts and the LV is not found (it's either been removed by the snapshot routine or something else has caused it to be removed). Here is a snippet from the beginning of the snapshot log:
INFO: starting new backup job: vzdump 1224 104 107 106 300 1101 1401 108 303 119 113 122 123 124 125 114 1106 1118 129 128 117 131 133 132 --quiet 1 --mailnotification always --mailto sysadmin@xxx.com --mode snapshot --node prox-011 --compress lzo --storage snapnfs
INFO: Starting Backup of VM 104 (qemu)
INFO: status = running
INFO: update VM 104: -lock backup
ERROR: Backup of VM 104 failed - can't activate LV '/dev/VMDATA4/vm-104-disk-1': One or more specified logical volume(s) not found.
INFO: Starting Backup of VM 106 (qemu)
INFO: status = running
INFO: update VM 106: -lock backup
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating archive '/mnt/pve/snapnfs/dump/vzdump-qemu-106-2016_01_24-16_04_04.vma.lzo'
INFO: started backup task '21463138-534a-4793-bf78-464fe05e24f5'
INFO: status: 0% (56623104/85899345920), sparse 0% (34492416), duration 3, 18/7 MB/s
INFO: status: 1% (862846976/85899345920), sparse 0% (196857856), duration 45, 19/15 MB/s
INFO: status: 2% (1733885952/85899345920), sparse 0% (349478912), duration 87, 20/17 MB/s
INFO: status: 3% (2585329664/85899345920), sparse 0% (560578560), duration 127, 21/16 MB/
As you can see VM 104 gets locked and then fails cos it's LV is not there.
Backup storage is a basic NFS server using Equallogic SAN as storage. These snapshots have been working in this way for well over a year without any problems, but suddenly in the last week I've has 3 instances of this. 2 on Proxmox 4.1 and the example above was on Proxmox 3.4 so it's not version-specific.
VM storage is from an Equallogic SAN group
If I do an fdisk /dev/mapper/VMDATA4-vm--104--disk--1 the disk appears to be there but there is not partition information at all, as if it's been wiped.
This has caused me to rebuild several VM's from scratch so it's fairly annoying.
Anyone seen anything like this before?
Thanks
Chris.
This has happened to me 3 times in the past week and has never been a problem before. Am a little baffled as to what is causing it.
Basically the scheduled vzdump snapshot starts and the LV is not found (it's either been removed by the snapshot routine or something else has caused it to be removed). Here is a snippet from the beginning of the snapshot log:
INFO: starting new backup job: vzdump 1224 104 107 106 300 1101 1401 108 303 119 113 122 123 124 125 114 1106 1118 129 128 117 131 133 132 --quiet 1 --mailnotification always --mailto sysadmin@xxx.com --mode snapshot --node prox-011 --compress lzo --storage snapnfs
INFO: Starting Backup of VM 104 (qemu)
INFO: status = running
INFO: update VM 104: -lock backup
ERROR: Backup of VM 104 failed - can't activate LV '/dev/VMDATA4/vm-104-disk-1': One or more specified logical volume(s) not found.
INFO: Starting Backup of VM 106 (qemu)
INFO: status = running
INFO: update VM 106: -lock backup
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating archive '/mnt/pve/snapnfs/dump/vzdump-qemu-106-2016_01_24-16_04_04.vma.lzo'
INFO: started backup task '21463138-534a-4793-bf78-464fe05e24f5'
INFO: status: 0% (56623104/85899345920), sparse 0% (34492416), duration 3, 18/7 MB/s
INFO: status: 1% (862846976/85899345920), sparse 0% (196857856), duration 45, 19/15 MB/s
INFO: status: 2% (1733885952/85899345920), sparse 0% (349478912), duration 87, 20/17 MB/s
INFO: status: 3% (2585329664/85899345920), sparse 0% (560578560), duration 127, 21/16 MB/
As you can see VM 104 gets locked and then fails cos it's LV is not there.
Backup storage is a basic NFS server using Equallogic SAN as storage. These snapshots have been working in this way for well over a year without any problems, but suddenly in the last week I've has 3 instances of this. 2 on Proxmox 4.1 and the example above was on Proxmox 3.4 so it's not version-specific.
VM storage is from an Equallogic SAN group
If I do an fdisk /dev/mapper/VMDATA4-vm--104--disk--1 the disk appears to be there but there is not partition information at all, as if it's been wiped.
This has caused me to rebuild several VM's from scratch so it's fairly annoying.
Anyone seen anything like this before?
Thanks
Chris.
Last edited: