Backup failure

K

k0nsl

Member
Nov 17, 2013
15
0
21
Denmark, Slangerup
k0nsl.org
I've tried to backup one KVM for several days and it always end bad with a log entry such as the following:

Code: 
Nov 02 10:25:54 INFO: Starting Backup of VM 104 (qemu)
Nov 02 10:25:54 INFO: status = running
Nov 02 10:25:55 INFO: update VM 104: -lock backup
Nov 02 10:25:55 INFO: VM Name: srv51-kvm
Nov 02 10:25:55 INFO: include disk 'virtio0' 'storage:vm-104-disk-1'
Nov 02 10:25:55 INFO: include disk 'virtio1' 'storage:vm-104-disk-2'
Nov 02 10:25:55 INFO: backup mode: snapshot
Nov 02 10:25:55 INFO: ionice priority: 7
Nov 02 10:25:55 INFO: pending configuration changes found (not included into backup)
Nov 02 10:25:56 INFO: creating archive '/mnt/pve/Gluster01/dump/vzdump-qemu-104-2016_11_02-10_25_54.vma.gz'
Nov 02 10:25:56 INFO: started backup task '38c0ef86-ae59-41c7-aa85-c1131c6da678'
Nov 02 10:25:59 INFO: status: 0% (254672896/1395864371200), sparse 0% (38039552), duration 3, 84/72 MB/s
Nov 02 10:28:02 INFO: status: 1% (14035976192/1395864371200), sparse 0% (2577686528), duration 126, 112/91 MB/s
Nov 02 10:31:21 INFO: status: 2% (27961065472/1395864371200), sparse 0% (3589496832), duration 325, 69/64 MB/s
Nov 02 10:34:15 INFO: status: 3% (41905618944/1395864371200), sparse 0% (4800688128), duration 499, 80/73 MB/s
Nov 02 10:37:07 INFO: status: 4% (55902601216/1395864371200), sparse 0% (7773581312), duration 671, 81/64 MB/s
Nov 02 10:40:23 INFO: status: 5% (69834375168/1395864371200), sparse 0% (8411758592), duration 867, 71/67 MB/s
Nov 02 10:43:47 INFO: status: 6% (83801210880/1395864371200), sparse 0% (9264824320), duration 1071, 68/64 MB/s
Nov 02 10:46:54 INFO: status: 7% (97748975616/1395864371200), sparse 0% (9663406080), duration 1258, 74/72 MB/s
Nov 02 10:49:20 INFO: status: 8% (111721578496/1395864371200), sparse 0% (9895198720), duration 1404, 95/94 MB/s
Nov 02 10:52:31 INFO: status: 9% (125656825856/1395864371200), sparse 0% (10142330880), duration 1595, 72/71 MB/s
Nov 02 10:57:45 INFO: status: 10% (139674779648/1395864371200), sparse 0% (10558214144), duration 1909, 44/43 MB/s
Nov 02 11:00:14 INFO: status: 11% (153552814080/1395864371200), sparse 0% (11356491776), duration 2058, 93/87 MB/s
Nov 02 11:02:55 INFO: status: 12% (167548616704/1395864371200), sparse 0% (12643295232), duration 2219, 86/78 MB/s
Nov 02 11:05:44 INFO: status: 13% (181480194048/1395864371200), sparse 1% (14067425280), duration 2388, 82/74 MB/s
Nov 02 11:09:00 INFO: status: 14% (195468787712/1395864371200), sparse 1% (14918557696), duration 2584, 71/67 MB/s
Nov 02 11:12:50 INFO: status: 15% (209425661952/1395864371200), sparse 1% (15529914368), duration 2814, 60/58 MB/s
Nov 02 11:15:32 INFO: status: 16% (223360843776/1395864371200), sparse 1% (15826960384), duration 2976, 86/84 MB/s
Nov 02 11:16:12 INFO: status: 16% (225911373824/1395864371200), sparse 1% (15860338688), duration 3016, 63/62 MB/s
Nov 02 11:16:12 ERROR: job failed with err -5 - Input/output error
Nov 02 11:16:12 INFO: aborting backup job
Nov 02 11:16:17 ERROR: Backup of VM 104 failed - job failed with err -5 - Input/output error

There is 1.8T available in backup location so I don't understand what the problem could be. I never had this problem before.

Any ideas? I'm running latest version available in the "non-subscription" repository. The output of "pveperf" shows:
Code: 
root@edelweiss:~# pveperf
CPU BOGOMIPS:      54397.20
REGEX/SECOND:      2470805
HD SIZE:           94.37 GB (/dev/dm-0)
BUFFERED READS:    19.89 MB/sec
AVERAGE SEEK TIME: 20.02 ms
FSYNCS/SECOND:     22.68
DNS EXT:           211.67 ms
DNS INT:           142.13 ms (k0nsl.org)

Thanks in advance for any hints on how to solve this. I really need to get this backed up, ASAP.

Best wishes,
-k0nsl
 
I have figured out why it fails. The disk is rapidly failing.

Here are some additional information:

Code: 
root@edelweiss:~# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda3
  VG Name               pve
  PV Size               1.82 TiB / not usable 1.07 MiB
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              476900
  Free PE               4039
  Allocated PE          472861
  PV UUID               QVyybl-qwjB-QTJC-f8gj-akbz-KpYC-zS5aYK
   
root@edelweiss:~# vgdisplay
  --- Volume group ---
  VG Name               pve
  System ID            
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  39
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                9
  Open LV               4
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               1.82 TiB
  PE Size               4.00 MiB
  Total PE              476900
  Alloc PE / Size       472861 / 1.80 TiB
  Free  PE / Size       4039 / 15.78 GiB
  VG UUID               RFhNxU-uj1x-D0Pi-EAxJ-T74u-ovCg-RBuCTM
   
root@edelweiss:~# lvdisplay
  --- Logical volume ---
  LV Path                /dev/pve/swap
  LV Name                swap
  VG Name                pve
  LV UUID                DTG56N-L30R-Xqlb-wukp-V6fb-1y8O-vYA3e8
  LV Write Access        read/write
  LV Creation host, time proxmox, 2016-09-20 22:15:15 -0900
  LV Status              available
  # open                 2
  LV Size                8.00 GiB
  Current LE             2048
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:1
   
  --- Logical volume ---
  LV Path                /dev/pve/root
  LV Name                root
  VG Name                pve
  LV UUID                GDwdZa-mOHR-vVQn-0Bsf-ePlp-5yL3-ygX91o
  LV Write Access        read/write
  LV Creation host, time proxmox, 2016-09-20 22:15:16 -0900
  LV Status              available
  # open                 1
  LV Size                96.00 GiB
  Current LE             24576
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:0
   
  --- Logical volume ---
  LV Name                data
  VG Name                pve
  LV UUID                x0T9HD-6htn-B4Eg-KB5Z-rmsd-GQH7-5kcOLy
  LV Write Access        read/write
  LV Creation host, time proxmox, 2016-09-20 22:15:17 -0900
  LV Pool metadata       data_tmeta
  LV Pool data           data_tdata
  LV Status              available
  # open                 7
  LV Size                1.70 TiB
  Allocated pool data    55.06%
  Allocated metadata     26.63%
  Current LE             446181
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:4
   
  --- Logical volume ---
  LV Path                /dev/pve/vm-100-disk-1
  LV Name                vm-100-disk-1
  VG Name                pve
  LV UUID                gdxGOM-Pcsa-Suf4-66Y3-fp6H-bz9H-GErG36
  LV Write Access        read/write
  LV Creation host, time edelweiss, 2016-09-21 13:11:06 -0900
  LV Pool name           data
  LV Status              available
  # open                 0
  LV Size                45.00 GiB
  Mapped size            100.00%
  Current LE             11520
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:6
   
  --- Logical volume ---
  LV Path                /dev/pve/vm-102-disk-1
  LV Name                vm-102-disk-1
  VG Name                pve
  LV UUID                dVbn0I-owWP-F3pL-5Nsy-OlTH-b0F9-QDjwHH
  LV Write Access        read/write
  LV Creation host, time edelweiss, 2016-09-22 05:32:47 -0900
  LV Pool name           data
  LV Status              available
  # open                 0
  LV Size                10.00 GiB
  Mapped size            42.27%
  Current LE             2560
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:7
   
  --- Logical volume ---
  LV Path                /dev/pve/vm-103-disk-1
  LV Name                vm-103-disk-1
  VG Name                pve
  LV UUID                jUxeOI-XCzT-aDf1-0Lej-mJ69-kAdy-sRC9FX
  LV Write Access        read/write
  LV Creation host, time edelweiss, 2016-09-23 23:56:20 -0900
  LV Pool name           data
  LV Status              available
  # open                 0
  LV Size                10.00 GiB
  Mapped size            25.68%
  Current LE             2560
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:8
   
  --- Logical volume ---
  LV Path                /dev/pve/vm-104-disk-1
  LV Name                vm-104-disk-1
  VG Name                pve
  LV UUID                ZussfF-8edD-fthO-DA2O-xr9J-HOOD-Wc0TIe
  LV Write Access        read/write
  LV Creation host, time edelweiss, 2016-09-28 13:04:18 -0900
  LV Pool name           data
  LV Status              available
  # open                 1
  LV Size                800.00 GiB
  Mapped size            92.25%
  Current LE             204800
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:9
   
  --- Logical volume ---
  LV Path                /dev/pve/vm-104-disk-2
  LV Name                vm-104-disk-2
  VG Name                pve
  LV UUID                9b71NQ-ZNI2-t5Jx-d9pY-DC9e-U1z3-1JGFgm
  LV Write Access        read/write
  LV Creation host, time edelweiss, 2016-09-28 13:04:19 -0900
  LV Pool name           data
  LV Status              available
  # open                 1
  LV Size                500.00 GiB
  Mapped size            27.56%
  Current LE             128000
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:10
   
  --- Logical volume ---
  LV Path                /dev/pve/vm-104-disk-3
  LV Name                vm-104-disk-3
  VG Name                pve
  LV UUID                Uxo23I-MJYj-AfNI-tTyO-PdZf-l9wU-cfMcuJ
  LV Write Access        read/write
  LV Creation host, time edelweiss, 2016-11-03 08:12:33 -0900
  LV Pool name           data
  LV Status              available
  # open                 0
  LV Size                500.00 GiB
  Mapped size            6.40%
  Current LE             128000
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:11
   
root@edelweiss:~#

I have only two logical volumes on that disk which I care about and those are "/dev/pve/vm-104-disk-1" and "/dev/pve/vm-104-disk-2". Can somebody recommend a safe way to safe them? I have another disk mounted which is functioning and could be used for that purpose. But what is the best/safest way to do it?

Any help is appreciated.

Best wishes,
-k0nsl
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom