Hi,
i've a problem during a backup job on one of my VM.
The Job aborts and VM goes down.
Here the log (problem with VM 101)
Why the VM 101 is not running during backup job?
How do I prevent VM 101 from being unreachable in this situation?
Thanks
i've a problem during a backup job on one of my VM.
The Job aborts and VM goes down.
Here the log (problem with VM 101)
Code:
INFO: starting new backup job: vzdump --mailto ****@******.com --mailnotification always --all 1 --compress zstd --mode snapshot --storage bk007 --quiet 1
INFO: Starting Backup of VM 100 (qemu)
INFO: Backup started at 2021-09-16 05:00:02
INFO: status = running
INFO: VM Name: vmvnginx100
INFO: include disk 'scsi0' 'local:100/vm-100-disk-0.qcow2' 1000G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: pending configuration changes found (not included into backup)
INFO: creating vzdump archive '/mnt/pve/bk007/dump/vzdump-qemu-100-2021_09_16-05_00_02.vma.zst'
INFO: started backup task '530afbdc-c908-481b-b544-243d4e6d8540'
INFO: resuming VM again
INFO: 0% (148.6 MiB of 1000.0 GiB) in 3s, read: 49.5 MiB/s, write: 5.5 MiB/s
INFO: 1% (11.0 GiB of 1000.0 GiB) in 1m 7s, read: 174.4 MiB/s, write: 29.3 MiB/s
INFO: 2% (21.4 GiB of 1000.0 GiB) in 1m 14s, read: 1.5 GiB/s, write: 47.8 MiB/s
INFO: 3% (31.1 GiB of 1000.0 GiB) in 1m 20s, read: 1.6 GiB/s, write: 57.1 MiB/s
INFO: 4% (40.1 GiB of 1000.0 GiB) in 4m, read: 57.0 MiB/s, write: 19.5 MiB/s
INFO: 5% (50.1 GiB of 1000.0 GiB) in 8m 53s, read: 35.0 MiB/s, write: 34.3 MiB/s
INFO: 6% (60.0 GiB of 1000.0 GiB) in 12m 8s, read: 52.3 MiB/s, write: 50.5 MiB/s
INFO: 7% (70.0 GiB of 1000.0 GiB) in 15m 15s, read: 54.7 MiB/s, write: 51.8 MiB/s
INFO: 8% (80.0 GiB of 1000.0 GiB) in 17m 5s, read: 93.3 MiB/s, write: 75.5 MiB/s
INFO: 9% (90.1 GiB of 1000.0 GiB) in 19m 38s, read: 67.3 MiB/s, write: 37.1 MiB/s
INFO: 10% (100.0 GiB of 1000.0 GiB) in 22m 42s, read: 55.3 MiB/s, write: 48.1 MiB/s
INFO: 11% (110.0 GiB of 1000.0 GiB) in 24m 59s, read: 74.6 MiB/s, write: 53.1 MiB/s
INFO: 12% (120.0 GiB of 1000.0 GiB) in 26m 42s, read: 99.6 MiB/s, write: 55.3 MiB/s
INFO: 13% (130.1 GiB of 1000.0 GiB) in 28m 56s, read: 76.7 MiB/s, write: 67.8 MiB/s
INFO: 14% (140.0 GiB of 1000.0 GiB) in 31m, read: 82.1 MiB/s, write: 74.2 MiB/s
INFO: 15% (150.5 GiB of 1000.0 GiB) in 31m 36s, read: 298.8 MiB/s, write: 75.4 MiB/s
INFO: 16% (161.0 GiB of 1000.0 GiB) in 31m 45s, read: 1.2 GiB/s, write: 58.7 MiB/s
INFO: 17% (170.8 GiB of 1000.0 GiB) in 31m 54s, read: 1.1 GiB/s, write: 74.6 MiB/s
INFO: 18% (181.1 GiB of 1000.0 GiB) in 32m 19s, read: 421.1 MiB/s, write: 44.5 MiB/s
INFO: 19% (190.1 GiB of 1000.0 GiB) in 32m 33s, read: 657.9 MiB/s, write: 84.1 MiB/s
INFO: 20% (200.4 GiB of 1000.0 GiB) in 33m 15s, read: 251.4 MiB/s, write: 68.9 MiB/s
INFO: 21% (218.8 GiB of 1000.0 GiB) in 33m 18s, read: 6.1 GiB/s, write: 0 B/s
INFO: 24% (240.7 GiB of 1000.0 GiB) in 33m 21s, read: 7.3 GiB/s, write: 0 B/s
INFO: 26% (260.5 GiB of 1000.0 GiB) in 33m 24s, read: 6.6 GiB/s, write: 0 B/s
INFO: 28% (281.6 GiB of 1000.0 GiB) in 33m 27s, read: 7.1 GiB/s, write: 0 B/s
INFO: 30% (302.1 GiB of 1000.0 GiB) in 33m 30s, read: 6.8 GiB/s, write: 0 B/s
INFO: 32% (321.8 GiB of 1000.0 GiB) in 33m 33s, read: 6.6 GiB/s, write: 0 B/s
INFO: 33% (339.1 GiB of 1000.0 GiB) in 33m 36s, read: 5.8 GiB/s, write: 0 B/s
INFO: 35% (359.1 GiB of 1000.0 GiB) in 33m 39s, read: 6.7 GiB/s, write: 0 B/s
INFO: 38% (380.8 GiB of 1000.0 GiB) in 33m 42s, read: 7.2 GiB/s, write: 0 B/s
INFO: 40% (400.4 GiB of 1000.0 GiB) in 33m 45s, read: 6.5 GiB/s, write: 0 B/s
INFO: 41% (419.4 GiB of 1000.0 GiB) in 33m 48s, read: 6.3 GiB/s, write: 0 B/s
INFO: 43% (435.6 GiB of 1000.0 GiB) in 33m 51s, read: 5.4 GiB/s, write: 0 B/s
INFO: 45% (454.5 GiB of 1000.0 GiB) in 33m 54s, read: 6.3 GiB/s, write: 0 B/s
INFO: 47% (474.1 GiB of 1000.0 GiB) in 33m 57s, read: 6.5 GiB/s, write: 0 B/s
INFO: 49% (492.2 GiB of 1000.0 GiB) in 34m, read: 6.0 GiB/s, write: 0 B/s
INFO: 51% (511.5 GiB of 1000.0 GiB) in 34m 3s, read: 6.4 GiB/s, write: 0 B/s
INFO: 52% (528.3 GiB of 1000.0 GiB) in 34m 6s, read: 5.6 GiB/s, write: 0 B/s
INFO: 54% (548.8 GiB of 1000.0 GiB) in 34m 9s, read: 6.8 GiB/s, write: 0 B/s
INFO: 56% (566.7 GiB of 1000.0 GiB) in 34m 12s, read: 6.0 GiB/s, write: 0 B/s
INFO: 58% (586.8 GiB of 1000.0 GiB) in 34m 15s, read: 6.7 GiB/s, write: 0 B/s
INFO: 60% (603.1 GiB of 1000.0 GiB) in 34m 18s, read: 5.4 GiB/s, write: 0 B/s
INFO: 62% (621.2 GiB of 1000.0 GiB) in 34m 21s, read: 6.0 GiB/s, write: 0 B/s
INFO: 63% (639.8 GiB of 1000.0 GiB) in 34m 24s, read: 6.2 GiB/s, write: 0 B/s
INFO: 65% (659.9 GiB of 1000.0 GiB) in 34m 27s, read: 6.7 GiB/s, write: 0 B/s
INFO: 67% (678.7 GiB of 1000.0 GiB) in 34m 30s, read: 6.3 GiB/s, write: 0 B/s
INFO: 69% (697.8 GiB of 1000.0 GiB) in 34m 33s, read: 6.4 GiB/s, write: 0 B/s
INFO: 71% (717.9 GiB of 1000.0 GiB) in 34m 36s, read: 6.7 GiB/s, write: 0 B/s
INFO: 73% (735.8 GiB of 1000.0 GiB) in 34m 39s, read: 6.0 GiB/s, write: 0 B/s
INFO: 74% (748.1 GiB of 1000.0 GiB) in 34m 42s, read: 4.1 GiB/s, write: 0 B/s
INFO: 76% (767.4 GiB of 1000.0 GiB) in 34m 45s, read: 6.4 GiB/s, write: 0 B/s
INFO: 78% (785.9 GiB of 1000.0 GiB) in 34m 48s, read: 6.2 GiB/s, write: 0 B/s
INFO: 80% (806.7 GiB of 1000.0 GiB) in 34m 51s, read: 6.9 GiB/s, write: 0 B/s
INFO: 82% (825.0 GiB of 1000.0 GiB) in 34m 54s, read: 6.1 GiB/s, write: 0 B/s
INFO: 84% (842.8 GiB of 1000.0 GiB) in 34m 57s, read: 5.9 GiB/s, write: 0 B/s
INFO: 86% (860.3 GiB of 1000.0 GiB) in 35m, read: 5.8 GiB/s, write: 0 B/s
INFO: 87% (878.3 GiB of 1000.0 GiB) in 35m 3s, read: 6.0 GiB/s, write: 0 B/s
INFO: 89% (896.7 GiB of 1000.0 GiB) in 35m 6s, read: 6.1 GiB/s, write: 0 B/s
INFO: 91% (912.7 GiB of 1000.0 GiB) in 35m 9s, read: 5.3 GiB/s, write: 0 B/s
INFO: 92% (929.6 GiB of 1000.0 GiB) in 35m 12s, read: 5.6 GiB/s, write: 0 B/s
INFO: 94% (946.1 GiB of 1000.0 GiB) in 35m 15s, read: 5.5 GiB/s, write: 0 B/s
INFO: 96% (961.6 GiB of 1000.0 GiB) in 35m 19s, read: 3.9 GiB/s, write: 0 B/s
INFO: 97% (979.8 GiB of 1000.0 GiB) in 35m 22s, read: 6.1 GiB/s, write: 0 B/s
INFO: 99% (997.3 GiB of 1000.0 GiB) in 35m 25s, read: 5.8 GiB/s, write: 0 B/s
INFO: 100% (1000.0 GiB of 1000.0 GiB) in 35m 26s, read: 2.7 GiB/s, write: 8.0 KiB/s
INFO: backup is sparse: 903.86 GiB (90%) total zero data
INFO: transferred 1000.00 GiB in 2126 seconds (481.7 MiB/s)
INFO: archive file size: 54.13GB
INFO: removing backup 'bk007:backup/vzdump-qemu-100-2021_09_13-23_30_02.vma.zst'
INFO: Finished Backup of VM 100 (00:35:43)
INFO: Backup finished at 2021-09-16 05:35:45
INFO: Starting Backup of VM 101 (qemu)
INFO: Backup started at 2021-09-16 05:35:46
INFO: status = running
INFO: VM Name: vmvmysql101
INFO: include disk 'scsi0' 'local:101/vm-101-disk-0.qcow2' 1000G
INFO: include disk 'scsi1' 'local:101/vm-101-disk-1.raw' 100G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating vzdump archive '/mnt/pve/bk007/dump/vzdump-qemu-101-2021_09_16-05_35_45.vma.zst'
INFO: started backup task 'a214a054-a034-47ef-bfe4-1229b1f42302'
INFO: resuming VM again
INFO: 2% (27.7 GiB of 1.1 TiB) in 3s, read: 9.2 GiB/s, write: 0 B/s
INFO: 5% (56.0 GiB of 1.1 TiB) in 6s, read: 9.4 GiB/s, write: 0 B/s
INFO: 7% (84.1 GiB of 1.1 TiB) in 9s, read: 9.4 GiB/s, write: 0 B/s
INFO: 9% (100.3 GiB of 1.1 TiB) in 12s, read: 5.4 GiB/s, write: 43.8 MiB/s
INFO: 10% (110.0 GiB of 1.1 TiB) in 2m 22s, read: 76.7 MiB/s, write: 66.8 MiB/s
INFO: 11% (121.1 GiB of 1.1 TiB) in 4m 14s, read: 101.2 MiB/s, write: 95.7 MiB/s
INFO: 12% (132.1 GiB of 1.1 TiB) in 5m 58s, read: 108.2 MiB/s, write: 105.3 MiB/s
INFO: 13% (143.1 GiB of 1.1 TiB) in 8m 52s, read: 64.7 MiB/s, write: 61.5 MiB/s
INFO: 14% (154.0 GiB of 1.1 TiB) in 12m 17s, read: 54.7 MiB/s, write: 50.7 MiB/s
INFO: 15% (165.2 GiB of 1.1 TiB) in 14m 10s, read: 101.1 MiB/s, write: 92.5 MiB/s
INFO: 16% (176.1 GiB of 1.1 TiB) in 15m 37s, read: 128.1 MiB/s, write: 117.1 MiB/s
INFO: 17% (187.1 GiB of 1.1 TiB) in 17m 18s, read: 111.6 MiB/s, write: 102.6 MiB/s
INFO: 18% (198.1 GiB of 1.1 TiB) in 18m 55s, read: 116.6 MiB/s, write: 109.6 MiB/s
INFO: 19% (209.1 GiB of 1.1 TiB) in 20m 41s, read: 106.6 MiB/s, write: 98.3 MiB/s
INFO: 20% (220.1 GiB of 1.1 TiB) in 22m 23s, read: 109.5 MiB/s, write: 104.5 MiB/s
INFO: 21% (231.2 GiB of 1.1 TiB) in 23m 57s, read: 121.4 MiB/s, write: 113.3 MiB/s
INFO: 22% (242.1 GiB of 1.1 TiB) in 25m 35s, read: 114.1 MiB/s, write: 108.7 MiB/s
INFO: 23% (253.2 GiB of 1.1 TiB) in 27m 10s, read: 119.2 MiB/s, write: 112.5 MiB/s
INFO: 24% (264.0 GiB of 1.1 TiB) in 28m 48s, read: 113.5 MiB/s, write: 107.6 MiB/s
INFO: 25% (275.0 GiB of 1.1 TiB) in 30m 51s, read: 91.6 MiB/s, write: 87.6 MiB/s
INFO: 26% (286.1 GiB of 1.1 TiB) in 33m 24s, read: 73.8 MiB/s, write: 72.3 MiB/s
INFO: 27% (297.0 GiB of 1.1 TiB) in 35m 24s, read: 93.3 MiB/s, write: 89.9 MiB/s
INFO: 28% (308.0 GiB of 1.1 TiB) in 37m 13s, read: 103.6 MiB/s, write: 97.7 MiB/s
INFO: 29% (319.2 GiB of 1.1 TiB) in 38m 53s, read: 113.9 MiB/s, write: 106.3 MiB/s
INFO: 30% (330.0 GiB of 1.1 TiB) in 40m 42s, read: 102.2 MiB/s, write: 96.4 MiB/s
INFO: 31% (341.2 GiB of 1.1 TiB) in 42m 35s, read: 100.9 MiB/s, write: 94.0 MiB/s
INFO: 32% (352.0 GiB of 1.1 TiB) in 44m 14s, read: 111.9 MiB/s, write: 105.5 MiB/s
INFO: 33% (363.1 GiB of 1.1 TiB) in 46m 7s, read: 100.2 MiB/s, write: 94.2 MiB/s
INFO: 34% (374.1 GiB of 1.1 TiB) in 47m 39s, read: 123.1 MiB/s, write: 115.7 MiB/s
INFO: 35% (385.2 GiB of 1.1 TiB) in 49m 26s, read: 105.8 MiB/s, write: 98.5 MiB/s
INFO: 36% (396.0 GiB of 1.1 TiB) in 52m 57s, read: 52.5 MiB/s, write: 49.5 MiB/s
INFO: 37% (407.0 GiB of 1.1 TiB) in 57m 25s, read: 42.0 MiB/s, write: 39.6 MiB/s
INFO: 38% (418.0 GiB of 1.1 TiB) in 1h 2m 45s, read: 35.2 MiB/s, write: 32.8 MiB/s
INFO: 39% (429.1 GiB of 1.1 TiB) in 1h 8m 10s, read: 35.1 MiB/s, write: 32.6 MiB/s
INFO: 40% (440.0 GiB of 1.1 TiB) in 1h 13m 55s, read: 32.3 MiB/s, write: 30.4 MiB/s
INFO: 41% (451.1 GiB of 1.1 TiB) in 1h 19m, read: 37.3 MiB/s, write: 34.8 MiB/s
INFO: 42% (462.0 GiB of 1.1 TiB) in 1h 24m 8s, read: 36.2 MiB/s, write: 34.1 MiB/s
INFO: 43% (479.0 GiB of 1.1 TiB) in 1h 25m 38s, read: 193.0 MiB/s, write: 15.1 MiB/s
INFO: 44% (487.2 GiB of 1.1 TiB) in 1h 25m 42s, read: 2.0 GiB/s, write: 5.0 KiB/s
INFO: 45% (497.0 GiB of 1.1 TiB) in 1h 25m 45s, read: 3.3 GiB/s, write: 174.7 KiB/s
INFO: 46% (513.0 GiB of 1.1 TiB) in 1h 25m 48s, read: 5.3 GiB/s, write: 10.7 KiB/s
INFO: 47% (522.9 GiB of 1.1 TiB) in 1h 25m 51s, read: 3.3 GiB/s, write: 5.3 KiB/s
INFO: 48% (532.2 GiB of 1.1 TiB) in 1h 25m 54s, read: 3.1 GiB/s, write: 6.7 KiB/s
INFO: 49% (541.3 GiB of 1.1 TiB) in 1h 25m 57s, read: 3.0 GiB/s, write: 6.7 KiB/s
INFO: 50% (554.8 GiB of 1.1 TiB) in 1h 26m, read: 4.5 GiB/s, write: 8.0 KiB/s
INFO: 51% (566.8 GiB of 1.1 TiB) in 1h 26m 3s, read: 4.0 GiB/s, write: 8.0 KiB/s
INFO: 52% (577.8 GiB of 1.1 TiB) in 1h 26m 6s, read: 3.7 GiB/s, write: 8.0 KiB/s
INFO: 53% (586.8 GiB of 1.1 TiB) in 1h 26m 10s, read: 2.3 GiB/s, write: 4.0 KiB/s
INFO: 54% (596.4 GiB of 1.1 TiB) in 1h 26m 13s, read: 3.2 GiB/s, write: 6.7 KiB/s
INFO: 55% (605.7 GiB of 1.1 TiB) in 1h 26m 20s, read: 1.3 GiB/s, write: 2.9 KiB/s
INFO: 56% (616.1 GiB of 1.1 TiB) in 1h 26m 24s, read: 2.6 GiB/s, write: 5.0 KiB/s
INFO: 57% (630.5 GiB of 1.1 TiB) in 1h 26m 35s, read: 1.3 GiB/s, write: 2.5 KiB/s
INFO: 58% (646.0 GiB of 1.1 TiB) in 1h 26m 38s, read: 5.2 GiB/s, write: 10.7 KiB/s
INFO: 59% (653.4 GiB of 1.1 TiB) in 1h 26m 41s, read: 2.4 GiB/s, write: 5.3 KiB/s
INFO: 60% (664.7 GiB of 1.1 TiB) in 1h 26m 44s, read: 3.8 GiB/s, write: 6.7 KiB/s
INFO: 61% (676.1 GiB of 1.1 TiB) in 1h 26m 47s, read: 3.8 GiB/s, write: 8.0 KiB/s
INFO: 62% (684.8 GiB of 1.1 TiB) in 1h 26m 51s, read: 2.2 GiB/s, write: 4.0 KiB/s
INFO: 63% (699.4 GiB of 1.1 TiB) in 1h 26m 54s, read: 4.9 GiB/s, write: 10.7 KiB/s
INFO: 64% (711.9 GiB of 1.1 TiB) in 1h 26m 57s, read: 4.2 GiB/s, write: 8.0 KiB/s
INFO: 65% (715.9 GiB of 1.1 TiB) in 1h 27m, read: 1.3 GiB/s, write: 2.7 KiB/s
INFO: 66% (730.1 GiB of 1.1 TiB) in 1h 27m 5s, read: 2.8 GiB/s, write: 5.6 KiB/s
INFO: 67% (744.5 GiB of 1.1 TiB) in 1h 27m 8s, read: 4.8 GiB/s, write: 9.3 KiB/s
INFO: 68% (757.1 GiB of 1.1 TiB) in 1h 27m 12s, read: 3.2 GiB/s, write: 7.0 KiB/s
INFO: 70% (772.5 GiB of 1.1 TiB) in 1h 27m 16s, read: 3.8 GiB/s, write: 7.0 KiB/s
INFO: 71% (781.4 GiB of 1.1 TiB) in 1h 27m 24s, read: 1.1 GiB/s, write: 2.5 KiB/s
ERROR: VM 101 not running
INFO: aborting backup job
ERROR: VM 101 not running
ERROR: Backup of VM 101 failed - VM 101 not running
INFO: Failed at 2021-09-16 07:04:04
INFO: Backup job finished with errors
TASK ERROR: job errorsWhy the VM 101 is not running during backup job?
How do I prevent VM 101 from being unreachable in this situation?
Thanks
