Can U Help me about this eror?
Log file:
Proxmox
Virtual Environment 7.4-16
Search
Virtual Machine 601 (ZmnSrv) on node 'casta-pve'
pbscasta
Filter VMID
Folder View
Logs
INFO: starting new backup job: vzdump 601 --storage pbscasta --mode snapshot --node casta-pve --notes-template '{{guestname}}' --remove 0
INFO: Starting Backup of VM 601 (qemu)
INFO: Backup started at 2023-09-07 14:22:17
INFO: status = running
INFO: VM Name: ZmnSrv
INFO: include disk 'virtio0' 'local-lvm:vm-601-disk-0' 300G
INFO: include disk 'virtio1' 'SamsungSSD:vm-601-disk-0' 1600G
INFO: include disk 'virtio2' 'local-lvm:vm-601-disk-1' 700G
INFO: include disk 'virtio3' 'local-lvm:vm-601-disk-2' 900G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/601/2023-09-07T11:22:17Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task '04d8cf2a-1973-4a0b-ba5c-6d49fb4187b1'
INFO: resuming VM again
INFO: virtio0: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: virtio1: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: virtio2: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: virtio3: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: 0% (408.0 MiB of 3.4 TiB) in 3s, read: 136.0 MiB/s, write: 118.7 MiB/s
INFO: 1% (35.0 GiB of 3.4 TiB) in 11m 39s, read: 51.0 MiB/s, write: 50.9 MiB/s
INFO: 2% (70.0 GiB of 3.4 TiB) in 22m 48s, read: 53.5 MiB/s, write: 53.0 MiB/s
INFO: 3% (105.0 GiB of 3.4 TiB) in 34m 28s, read: 51.2 MiB/s, write: 51.1 MiB/s
INFO: 4% (140.0 GiB of 3.4 TiB) in 44m 2s, read: 62.4 MiB/s, write: 62.4 MiB/s
INFO: 5% (175.1 GiB of 3.4 TiB) in 52m 23s, read: 71.6 MiB/s, write: 71.6 MiB/s
INFO: 6% (210.0 GiB of 3.4 TiB) in 1h 40s, read: 72.1 MiB/s, write: 72.1 MiB/s
INFO: 7% (245.1 GiB of 3.4 TiB) in 1h 9m 1s, read: 71.6 MiB/s, write: 71.6 MiB/s
INFO: 8% (280.1 GiB of 3.4 TiB) in 1h 17m 26s, read: 71.0 MiB/s, write: 71.0 MiB/s
INFO: 9% (315.0 GiB of 3.4 TiB) in 1h 25m 45s, read: 71.8 MiB/s, write: 71.8 MiB/s
INFO: 10% (350.0 GiB of 3.4 TiB) in 1h 34m 10s, read: 70.9 MiB/s, write: 70.9 MiB/s
INFO: 11% (385.0 GiB of 3.4 TiB) in 1h 42m 27s, read: 72.1 MiB/s, write: 72.1 MiB/s
INFO: 12% (420.0 GiB of 3.4 TiB) in 1h 50m 42s, read: 72.4 MiB/s, write: 72.4 MiB/s
INFO: 13% (455.0 GiB of 3.4 TiB) in 1h 59m 22s, read: 68.9 MiB/s, write: 68.9 MiB/s
INFO: 14% (490.0 GiB of 3.4 TiB) in 2h 7m 37s, read: 72.4 MiB/s, write: 72.4 MiB/s
INFO: 15% (525.0 GiB of 3.4 TiB) in 2h 16m 13s, read: 69.4 MiB/s, write: 69.4 MiB/s
INFO: 16% (560.1 GiB of 3.4 TiB) in 2h 26m 40s, read: 57.3 MiB/s, write: 56.7 MiB/s
INFO: 17% (595.0 GiB of 3.4 TiB) in 2h 38m 54s, read: 48.7 MiB/s, write: 48.7 MiB/s
INFO: 18% (630.0 GiB of 3.4 TiB) in 2h 51m 6s, read: 49.0 MiB/s, write: 49.0 MiB/s
INFO: 19% (665.0 GiB of 3.4 TiB) in 3h 2m 28s, read: 52.6 MiB/s, write: 52.3 MiB/s
INFO: 20% (700.0 GiB of 3.4 TiB) in 3h 13m 7s, read: 56.1 MiB/s, write: 55.8 MiB/s
INFO: 21% (735.0 GiB of 3.4 TiB) in 3h 22m 22s, read: 64.6 MiB/s, write: 64.4 MiB/s
INFO: 22% (770.1 GiB of 3.4 TiB) in 3h 32m, read: 62.1 MiB/s, write: 61.9 MiB/s
INFO: 23% (805.7 GiB of 3.4 TiB) in 3h 39m 30s, read: 81.2 MiB/s, write: 60.5 MiB/s
INFO: 24% (840.5 GiB of 3.4 TiB) in 3h 39m 49s, read: 1.8 GiB/s, write: 0 B/s
INFO: 25% (876.6 GiB of 3.4 TiB) in 3h 40m 9s, read: 1.8 GiB/s, write: 0 B/s
INFO: 26% (911.5 GiB of 3.4 TiB) in 3h 40m 29s, read: 1.7 GiB/s, write: 0 B/s
INFO: 27% (945.0 GiB of 3.4 TiB) in 3h 48m 50s, read: 68.6 MiB/s, write: 66.0 MiB/s
INFO: 28% (980.0 GiB of 3.4 TiB) in 3h 58m 12s, read: 63.8 MiB/s, write: 63.8 MiB/s
INFO: 29% (1015.0 GiB of 3.4 TiB) in 4h 7m 41s, read: 62.9 MiB/s, write: 60.6 MiB/s
INFO: 30% (1.0 TiB of 3.4 TiB) in 4h 16m 44s, read: 66.0 MiB/s, write: 63.6 MiB/s
INFO: 31% (1.1 TiB of 3.4 TiB) in 4h 25m 56s, read: 64.9 MiB/s, write: 62.1 MiB/s
INFO: 32% (1.1 TiB of 3.4 TiB) in 4h 36m 3s, read: 59.0 MiB/s, write: 59.0 MiB/s
INFO: 33% (1.1 TiB of 3.4 TiB) in 4h 46m 8s, read: 59.2 MiB/s, write: 59.2 MiB/s
INFO: 34% (1.2 TiB of 3.4 TiB) in 4h 56m 5s, read: 60.1 MiB/s, write: 60.0 MiB/s
INFO: 35% (1.2 TiB of 3.4 TiB) in 5h 6m 12s, read: 59.0 MiB/s, write: 59.0 MiB/s
INFO: 36% (1.2 TiB of 3.4 TiB) in 5h 15m 24s, read: 64.9 MiB/s, write: 58.9 MiB/s
INFO: 37% (1.3 TiB of 3.4 TiB) in 5h 22m 24s, read: 89.1 MiB/s, write: 67.0 MiB/s
INFO: 38% (1.3 TiB of 3.4 TiB) in 5h 23m 6s, read: 847.6 MiB/s, write: 50.8 MiB/s
INFO: 39% (1.3 TiB of 3.4 TiB) in 5h 23m 35s, read: 1.2 GiB/s, write: 15.7 MiB/s
INFO: 40% (1.4 TiB of 3.4 TiB) in 5h 23m 53s, read: 1.9 GiB/s, write: 21.8 MiB/s
INFO: 41% (1.4 TiB of 3.4 TiB) in 5h 24m 15s, read: 1.6 GiB/s, write: 10.0 MiB/s
INFO: 42% (1.4 TiB of 3.4 TiB) in 5h 24m 38s, read: 1.5 GiB/s, write: 5.6 MiB/s
INFO: 43% (1.5 TiB of 3.4 TiB) in 5h 25m 1s, read: 1.6 GiB/s, write: 2.3 MiB/s
INFO: 44% (1.5 TiB of 3.4 TiB) in 5h 25m 21s, read: 1.7 GiB/s, write: 1.6 MiB/s
INFO: 45% (1.5 TiB of 3.4 TiB) in 5h 25m 43s, read: 1.6 GiB/s, write: 14.9 MiB/s
INFO: 46% (1.6 TiB of 3.4 TiB) in 5h 26m 5s, read: 1.6 GiB/s, write: 5.5 MiB/s
INFO: 47% (1.6 TiB of 3.4 TiB) in 5h 34m 11s, read: 73.3 MiB/s, write: 61.4 MiB/s
INFO: 48% (1.6 TiB of 3.4 TiB) in 5h 43m 49s, read: 62.0 MiB/s, write: 61.9 MiB/s
INFO: 49% (1.7 TiB of 3.4 TiB) in 5h 54m 15s, read: 57.3 MiB/s, write: 57.3 MiB/s
INFO: 50% (1.7 TiB of 3.4 TiB) in 6h 3m 54s, read: 61.9 MiB/s, write: 61.6 MiB/s
INFO: 51% (1.7 TiB of 3.4 TiB) in 6h 13m 45s, read: 60.7 MiB/s, write: 60.6 MiB/s
INFO: 52% (1.8 TiB of 3.4 TiB) in 6h 23m 37s, read: 60.6 MiB/s, write: 60.5 MiB/s
INFO: 53% (1.8 TiB of 3.4 TiB) in 6h 33m 23s, read: 61.1 MiB/s, write: 61.1 MiB/s
INFO: 54% (1.8 TiB of 3.4 TiB) in 6h 42m 57s, read: 62.4 MiB/s, write: 55.9 MiB/s
INFO: 55% (1.9 TiB of 3.4 TiB) in 6h 52m 39s, read: 61.7 MiB/s, write: 61.7 MiB/s
INFO: 56% (1.9 TiB of 3.4 TiB) in 7h 2m 4s, read: 63.3 MiB/s, write: 63.3 MiB/s
INFO: 57% (1.9 TiB of 3.4 TiB) in 7h 12m 12s, read: 58.9 MiB/s, write: 58.9 MiB/s
INFO: 58% (2.0 TiB of 3.4 TiB) in 7h 23m 27s, read: 53.1 MiB/s, write: 52.8 MiB/s
INFO: 59% (2.0 TiB of 3.4 TiB) in 7h 34m 38s, read: 53.4 MiB/s, write: 53.3 MiB/s
INFO: 60% (2.1 TiB of 3.4 TiB) in 7h 46m 1s, read: 52.5 MiB/s, write: 52.4 MiB/s
INFO: 61% (2.1 TiB of 3.4 TiB) in 7h 55m 27s, read: 63.3 MiB/s, write: 63.2 MiB/s
INFO: 62% (2.1 TiB of 3.4 TiB) in 8h 4m, read: 69.8 MiB/s, write: 69.8 MiB/s
INFO: 63% (2.2 TiB of 3.4 TiB) in 8h 9m 28s, read: 109.3 MiB/s, write: 109.2 MiB/s
INFO: 64% (2.2 TiB of 3.4 TiB) in 8h 15m 18s, read: 102.4 MiB/s, write: 102.2 MiB/s
INFO: 65% (2.2 TiB of 3.4 TiB) in 8h 23m 9s, read: 76.2 MiB/s, write: 76.0 MiB/s
INFO: 66% (2.3 TiB of 3.4 TiB) in 8h 32m 21s, read: 64.9 MiB/s, write: 64.9 MiB/s
INFO: 67% (2.3 TiB of 3.4 TiB) in 8h 42m 10s, read: 60.8 MiB/s, write: 60.8 MiB/s
INFO: 68% (2.3 TiB of 3.4 TiB) in 8h 51m 32s, read: 63.7 MiB/s, write: 63.6 MiB/s
INFO: 69% (2.4 TiB of 3.4 TiB) in 9h 1m 21s, read: 60.9 MiB/s, write: 60.9 MiB/s
INFO: 70% (2.4 TiB of 3.4 TiB) in 9h 11m 3s, read: 61.5 MiB/s, write: 61.5 MiB/s
INFO: 71% (2.4 TiB of 3.4 TiB) in 9h 20m 59s, read: 60.2 MiB/s, write: 60.0 MiB/s
INFO: 72% (2.5 TiB of 3.4 TiB) in 9h 30m 30s, read: 62.7 MiB/s, write: 62.6 MiB/s
INFO: 73% (2.5 TiB of 3.4 TiB) in 9h 40m 2s, read: 62.7 MiB/s, write: 62.5 MiB/s
INFO: 74% (2.5 TiB of 3.4 TiB) in 9h 49m 41s, read: 61.9 MiB/s, write: 61.8 MiB/s
INFO: 75% (2.6 TiB of 3.4 TiB) in 9h 59m 23s, read: 61.5 MiB/s, write: 61.5 MiB/s
INFO: 76% (2.6 TiB of 3.4 TiB) in 10h 9m 30s, read: 59.0 MiB/s, write: 59.0 MiB/s
INFO: 77% (2.6 TiB of 3.4 TiB) in 10h 19m 8s, read: 62.0 MiB/s, write: 62.0 MiB/s
INFO: 78% (2.7 TiB of 3.4 TiB) in 10h 28m 27s, read: 64.1 MiB/s, write: 64.0 MiB/s
INFO: 79% (2.7 TiB of 3.4 TiB) in 10h 37m 1s, read: 69.8 MiB/s, write: 69.8 MiB/s
INFO: 80% (2.7 TiB of 3.4 TiB) in 10h 45m 48s, read: 67.9 MiB/s, write: 67.9 MiB/s
INFO: 81% (2.8 TiB of 3.4 TiB) in 10h 54m 46s, read: 66.7 MiB/s, write: 66.7 MiB/s
INFO: 82% (2.8 TiB of 3.4 TiB) in 11h 3m 42s, read: 66.9 MiB/s, write: 66.9 MiB/s
INFO: 83% (2.8 TiB of 3.4 TiB) in 11h 13m 34s, read: 60.5 MiB/s, write: 60.5 MiB/s
INFO: 84% (2.9 TiB of 3.4 TiB) in 11h 23m 8s, read: 62.4 MiB/s, write: 62.4 MiB/s
INFO: 85% (2.9 TiB of 3.4 TiB) in 11h 32m 14s, read: 65.7 MiB/s, write: 65.7 MiB/s
INFO: 86% (2.9 TiB of 3.4 TiB) in 11h 42m 9s, read: 60.2 MiB/s, write: 60.2 MiB/s
INFO: 87% (3.0 TiB of 3.4 TiB) in 11h 51m 55s, read: 61.2 MiB/s, write: 61.2 MiB/s
INFO: 88% (3.0 TiB of 3.4 TiB) in 12h 1m 41s, read: 61.2 MiB/s, write: 61.2 MiB/s
INFO: 89% (3.0 TiB of 3.4 TiB) in 12h 11m 18s, read: 62.1 MiB/s, write: 62.1 MiB/s
INFO: 90% (3.1 TiB of 3.4 TiB) in 12h 21m 23s, read: 59.2 MiB/s, write: 59.2 MiB/s
INFO: 91% (3.1 TiB of 3.4 TiB) in 12h 31m 15s, read: 60.6 MiB/s, write: 60.6 MiB/s
INFO: 92% (3.1 TiB of 3.4 TiB) in 12h 37m 27s, read: 96.4 MiB/s, write: 58.7 MiB/s
INFO: 93% (3.2 TiB of 3.4 TiB) in 12h 47m 29s, read: 59.5 MiB/s, write: 59.4 MiB/s
INFO: 93% (3.2 TiB of 3.4 TiB) in 12h 54m 36s, read: 56.1 MiB/s, write: 56.0 MiB/s
ERROR: backup write data failed: command error: write_data upload error: pipelined request failed: inserting chunk on store 'datastorecasta' failed for 072a0ef96c62d76a7a3cd1b24e5c7ecc9100c9d6fd87e828b820594eeee437d6 - Atomic rename failed for file "/mnt/datastore/datastorecasta/.chunks/072a/072a0ef96c62d76a7a3cd1b24e5c7ecc9100c9d6fd87e828b820594eeee437d6" - Structure needs cleaning (os error 117)
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 601 failed - backup write data failed: command error: write_data upload error: pipelined request failed: inserting chunk on store 'datastorecasta' failed for 072a0ef96c62d76a7a3cd1b24e5c7ecc9100c9d6fd87e828b820594eeee437d6 - Atomic rename failed for file "/mnt/datastore/datastorecasta/.chunks/072a/072a0ef96c62d76a7a3cd1b24e5c7ecc9100c9d6fd87e828b820594eeee437d6" - Structure needs cleaning (os error 117)
INFO: Failed at 2023-09-08 03:17:02
INFO: Backup job finished with errors
TASK ERROR: job errors
Log file:
Proxmox
Virtual Environment 7.4-16
Search
Virtual Machine 601 (ZmnSrv) on node 'casta-pve'
pbscasta
Filter VMID
Folder View
Logs
INFO: starting new backup job: vzdump 601 --storage pbscasta --mode snapshot --node casta-pve --notes-template '{{guestname}}' --remove 0
INFO: Starting Backup of VM 601 (qemu)
INFO: Backup started at 2023-09-07 14:22:17
INFO: status = running
INFO: VM Name: ZmnSrv
INFO: include disk 'virtio0' 'local-lvm:vm-601-disk-0' 300G
INFO: include disk 'virtio1' 'SamsungSSD:vm-601-disk-0' 1600G
INFO: include disk 'virtio2' 'local-lvm:vm-601-disk-1' 700G
INFO: include disk 'virtio3' 'local-lvm:vm-601-disk-2' 900G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/601/2023-09-07T11:22:17Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task '04d8cf2a-1973-4a0b-ba5c-6d49fb4187b1'
INFO: resuming VM again
INFO: virtio0: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: virtio1: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: virtio2: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: virtio3: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: 0% (408.0 MiB of 3.4 TiB) in 3s, read: 136.0 MiB/s, write: 118.7 MiB/s
INFO: 1% (35.0 GiB of 3.4 TiB) in 11m 39s, read: 51.0 MiB/s, write: 50.9 MiB/s
INFO: 2% (70.0 GiB of 3.4 TiB) in 22m 48s, read: 53.5 MiB/s, write: 53.0 MiB/s
INFO: 3% (105.0 GiB of 3.4 TiB) in 34m 28s, read: 51.2 MiB/s, write: 51.1 MiB/s
INFO: 4% (140.0 GiB of 3.4 TiB) in 44m 2s, read: 62.4 MiB/s, write: 62.4 MiB/s
INFO: 5% (175.1 GiB of 3.4 TiB) in 52m 23s, read: 71.6 MiB/s, write: 71.6 MiB/s
INFO: 6% (210.0 GiB of 3.4 TiB) in 1h 40s, read: 72.1 MiB/s, write: 72.1 MiB/s
INFO: 7% (245.1 GiB of 3.4 TiB) in 1h 9m 1s, read: 71.6 MiB/s, write: 71.6 MiB/s
INFO: 8% (280.1 GiB of 3.4 TiB) in 1h 17m 26s, read: 71.0 MiB/s, write: 71.0 MiB/s
INFO: 9% (315.0 GiB of 3.4 TiB) in 1h 25m 45s, read: 71.8 MiB/s, write: 71.8 MiB/s
INFO: 10% (350.0 GiB of 3.4 TiB) in 1h 34m 10s, read: 70.9 MiB/s, write: 70.9 MiB/s
INFO: 11% (385.0 GiB of 3.4 TiB) in 1h 42m 27s, read: 72.1 MiB/s, write: 72.1 MiB/s
INFO: 12% (420.0 GiB of 3.4 TiB) in 1h 50m 42s, read: 72.4 MiB/s, write: 72.4 MiB/s
INFO: 13% (455.0 GiB of 3.4 TiB) in 1h 59m 22s, read: 68.9 MiB/s, write: 68.9 MiB/s
INFO: 14% (490.0 GiB of 3.4 TiB) in 2h 7m 37s, read: 72.4 MiB/s, write: 72.4 MiB/s
INFO: 15% (525.0 GiB of 3.4 TiB) in 2h 16m 13s, read: 69.4 MiB/s, write: 69.4 MiB/s
INFO: 16% (560.1 GiB of 3.4 TiB) in 2h 26m 40s, read: 57.3 MiB/s, write: 56.7 MiB/s
INFO: 17% (595.0 GiB of 3.4 TiB) in 2h 38m 54s, read: 48.7 MiB/s, write: 48.7 MiB/s
INFO: 18% (630.0 GiB of 3.4 TiB) in 2h 51m 6s, read: 49.0 MiB/s, write: 49.0 MiB/s
INFO: 19% (665.0 GiB of 3.4 TiB) in 3h 2m 28s, read: 52.6 MiB/s, write: 52.3 MiB/s
INFO: 20% (700.0 GiB of 3.4 TiB) in 3h 13m 7s, read: 56.1 MiB/s, write: 55.8 MiB/s
INFO: 21% (735.0 GiB of 3.4 TiB) in 3h 22m 22s, read: 64.6 MiB/s, write: 64.4 MiB/s
INFO: 22% (770.1 GiB of 3.4 TiB) in 3h 32m, read: 62.1 MiB/s, write: 61.9 MiB/s
INFO: 23% (805.7 GiB of 3.4 TiB) in 3h 39m 30s, read: 81.2 MiB/s, write: 60.5 MiB/s
INFO: 24% (840.5 GiB of 3.4 TiB) in 3h 39m 49s, read: 1.8 GiB/s, write: 0 B/s
INFO: 25% (876.6 GiB of 3.4 TiB) in 3h 40m 9s, read: 1.8 GiB/s, write: 0 B/s
INFO: 26% (911.5 GiB of 3.4 TiB) in 3h 40m 29s, read: 1.7 GiB/s, write: 0 B/s
INFO: 27% (945.0 GiB of 3.4 TiB) in 3h 48m 50s, read: 68.6 MiB/s, write: 66.0 MiB/s
INFO: 28% (980.0 GiB of 3.4 TiB) in 3h 58m 12s, read: 63.8 MiB/s, write: 63.8 MiB/s
INFO: 29% (1015.0 GiB of 3.4 TiB) in 4h 7m 41s, read: 62.9 MiB/s, write: 60.6 MiB/s
INFO: 30% (1.0 TiB of 3.4 TiB) in 4h 16m 44s, read: 66.0 MiB/s, write: 63.6 MiB/s
INFO: 31% (1.1 TiB of 3.4 TiB) in 4h 25m 56s, read: 64.9 MiB/s, write: 62.1 MiB/s
INFO: 32% (1.1 TiB of 3.4 TiB) in 4h 36m 3s, read: 59.0 MiB/s, write: 59.0 MiB/s
INFO: 33% (1.1 TiB of 3.4 TiB) in 4h 46m 8s, read: 59.2 MiB/s, write: 59.2 MiB/s
INFO: 34% (1.2 TiB of 3.4 TiB) in 4h 56m 5s, read: 60.1 MiB/s, write: 60.0 MiB/s
INFO: 35% (1.2 TiB of 3.4 TiB) in 5h 6m 12s, read: 59.0 MiB/s, write: 59.0 MiB/s
INFO: 36% (1.2 TiB of 3.4 TiB) in 5h 15m 24s, read: 64.9 MiB/s, write: 58.9 MiB/s
INFO: 37% (1.3 TiB of 3.4 TiB) in 5h 22m 24s, read: 89.1 MiB/s, write: 67.0 MiB/s
INFO: 38% (1.3 TiB of 3.4 TiB) in 5h 23m 6s, read: 847.6 MiB/s, write: 50.8 MiB/s
INFO: 39% (1.3 TiB of 3.4 TiB) in 5h 23m 35s, read: 1.2 GiB/s, write: 15.7 MiB/s
INFO: 40% (1.4 TiB of 3.4 TiB) in 5h 23m 53s, read: 1.9 GiB/s, write: 21.8 MiB/s
INFO: 41% (1.4 TiB of 3.4 TiB) in 5h 24m 15s, read: 1.6 GiB/s, write: 10.0 MiB/s
INFO: 42% (1.4 TiB of 3.4 TiB) in 5h 24m 38s, read: 1.5 GiB/s, write: 5.6 MiB/s
INFO: 43% (1.5 TiB of 3.4 TiB) in 5h 25m 1s, read: 1.6 GiB/s, write: 2.3 MiB/s
INFO: 44% (1.5 TiB of 3.4 TiB) in 5h 25m 21s, read: 1.7 GiB/s, write: 1.6 MiB/s
INFO: 45% (1.5 TiB of 3.4 TiB) in 5h 25m 43s, read: 1.6 GiB/s, write: 14.9 MiB/s
INFO: 46% (1.6 TiB of 3.4 TiB) in 5h 26m 5s, read: 1.6 GiB/s, write: 5.5 MiB/s
INFO: 47% (1.6 TiB of 3.4 TiB) in 5h 34m 11s, read: 73.3 MiB/s, write: 61.4 MiB/s
INFO: 48% (1.6 TiB of 3.4 TiB) in 5h 43m 49s, read: 62.0 MiB/s, write: 61.9 MiB/s
INFO: 49% (1.7 TiB of 3.4 TiB) in 5h 54m 15s, read: 57.3 MiB/s, write: 57.3 MiB/s
INFO: 50% (1.7 TiB of 3.4 TiB) in 6h 3m 54s, read: 61.9 MiB/s, write: 61.6 MiB/s
INFO: 51% (1.7 TiB of 3.4 TiB) in 6h 13m 45s, read: 60.7 MiB/s, write: 60.6 MiB/s
INFO: 52% (1.8 TiB of 3.4 TiB) in 6h 23m 37s, read: 60.6 MiB/s, write: 60.5 MiB/s
INFO: 53% (1.8 TiB of 3.4 TiB) in 6h 33m 23s, read: 61.1 MiB/s, write: 61.1 MiB/s
INFO: 54% (1.8 TiB of 3.4 TiB) in 6h 42m 57s, read: 62.4 MiB/s, write: 55.9 MiB/s
INFO: 55% (1.9 TiB of 3.4 TiB) in 6h 52m 39s, read: 61.7 MiB/s, write: 61.7 MiB/s
INFO: 56% (1.9 TiB of 3.4 TiB) in 7h 2m 4s, read: 63.3 MiB/s, write: 63.3 MiB/s
INFO: 57% (1.9 TiB of 3.4 TiB) in 7h 12m 12s, read: 58.9 MiB/s, write: 58.9 MiB/s
INFO: 58% (2.0 TiB of 3.4 TiB) in 7h 23m 27s, read: 53.1 MiB/s, write: 52.8 MiB/s
INFO: 59% (2.0 TiB of 3.4 TiB) in 7h 34m 38s, read: 53.4 MiB/s, write: 53.3 MiB/s
INFO: 60% (2.1 TiB of 3.4 TiB) in 7h 46m 1s, read: 52.5 MiB/s, write: 52.4 MiB/s
INFO: 61% (2.1 TiB of 3.4 TiB) in 7h 55m 27s, read: 63.3 MiB/s, write: 63.2 MiB/s
INFO: 62% (2.1 TiB of 3.4 TiB) in 8h 4m, read: 69.8 MiB/s, write: 69.8 MiB/s
INFO: 63% (2.2 TiB of 3.4 TiB) in 8h 9m 28s, read: 109.3 MiB/s, write: 109.2 MiB/s
INFO: 64% (2.2 TiB of 3.4 TiB) in 8h 15m 18s, read: 102.4 MiB/s, write: 102.2 MiB/s
INFO: 65% (2.2 TiB of 3.4 TiB) in 8h 23m 9s, read: 76.2 MiB/s, write: 76.0 MiB/s
INFO: 66% (2.3 TiB of 3.4 TiB) in 8h 32m 21s, read: 64.9 MiB/s, write: 64.9 MiB/s
INFO: 67% (2.3 TiB of 3.4 TiB) in 8h 42m 10s, read: 60.8 MiB/s, write: 60.8 MiB/s
INFO: 68% (2.3 TiB of 3.4 TiB) in 8h 51m 32s, read: 63.7 MiB/s, write: 63.6 MiB/s
INFO: 69% (2.4 TiB of 3.4 TiB) in 9h 1m 21s, read: 60.9 MiB/s, write: 60.9 MiB/s
INFO: 70% (2.4 TiB of 3.4 TiB) in 9h 11m 3s, read: 61.5 MiB/s, write: 61.5 MiB/s
INFO: 71% (2.4 TiB of 3.4 TiB) in 9h 20m 59s, read: 60.2 MiB/s, write: 60.0 MiB/s
INFO: 72% (2.5 TiB of 3.4 TiB) in 9h 30m 30s, read: 62.7 MiB/s, write: 62.6 MiB/s
INFO: 73% (2.5 TiB of 3.4 TiB) in 9h 40m 2s, read: 62.7 MiB/s, write: 62.5 MiB/s
INFO: 74% (2.5 TiB of 3.4 TiB) in 9h 49m 41s, read: 61.9 MiB/s, write: 61.8 MiB/s
INFO: 75% (2.6 TiB of 3.4 TiB) in 9h 59m 23s, read: 61.5 MiB/s, write: 61.5 MiB/s
INFO: 76% (2.6 TiB of 3.4 TiB) in 10h 9m 30s, read: 59.0 MiB/s, write: 59.0 MiB/s
INFO: 77% (2.6 TiB of 3.4 TiB) in 10h 19m 8s, read: 62.0 MiB/s, write: 62.0 MiB/s
INFO: 78% (2.7 TiB of 3.4 TiB) in 10h 28m 27s, read: 64.1 MiB/s, write: 64.0 MiB/s
INFO: 79% (2.7 TiB of 3.4 TiB) in 10h 37m 1s, read: 69.8 MiB/s, write: 69.8 MiB/s
INFO: 80% (2.7 TiB of 3.4 TiB) in 10h 45m 48s, read: 67.9 MiB/s, write: 67.9 MiB/s
INFO: 81% (2.8 TiB of 3.4 TiB) in 10h 54m 46s, read: 66.7 MiB/s, write: 66.7 MiB/s
INFO: 82% (2.8 TiB of 3.4 TiB) in 11h 3m 42s, read: 66.9 MiB/s, write: 66.9 MiB/s
INFO: 83% (2.8 TiB of 3.4 TiB) in 11h 13m 34s, read: 60.5 MiB/s, write: 60.5 MiB/s
INFO: 84% (2.9 TiB of 3.4 TiB) in 11h 23m 8s, read: 62.4 MiB/s, write: 62.4 MiB/s
INFO: 85% (2.9 TiB of 3.4 TiB) in 11h 32m 14s, read: 65.7 MiB/s, write: 65.7 MiB/s
INFO: 86% (2.9 TiB of 3.4 TiB) in 11h 42m 9s, read: 60.2 MiB/s, write: 60.2 MiB/s
INFO: 87% (3.0 TiB of 3.4 TiB) in 11h 51m 55s, read: 61.2 MiB/s, write: 61.2 MiB/s
INFO: 88% (3.0 TiB of 3.4 TiB) in 12h 1m 41s, read: 61.2 MiB/s, write: 61.2 MiB/s
INFO: 89% (3.0 TiB of 3.4 TiB) in 12h 11m 18s, read: 62.1 MiB/s, write: 62.1 MiB/s
INFO: 90% (3.1 TiB of 3.4 TiB) in 12h 21m 23s, read: 59.2 MiB/s, write: 59.2 MiB/s
INFO: 91% (3.1 TiB of 3.4 TiB) in 12h 31m 15s, read: 60.6 MiB/s, write: 60.6 MiB/s
INFO: 92% (3.1 TiB of 3.4 TiB) in 12h 37m 27s, read: 96.4 MiB/s, write: 58.7 MiB/s
INFO: 93% (3.2 TiB of 3.4 TiB) in 12h 47m 29s, read: 59.5 MiB/s, write: 59.4 MiB/s
INFO: 93% (3.2 TiB of 3.4 TiB) in 12h 54m 36s, read: 56.1 MiB/s, write: 56.0 MiB/s
ERROR: backup write data failed: command error: write_data upload error: pipelined request failed: inserting chunk on store 'datastorecasta' failed for 072a0ef96c62d76a7a3cd1b24e5c7ecc9100c9d6fd87e828b820594eeee437d6 - Atomic rename failed for file "/mnt/datastore/datastorecasta/.chunks/072a/072a0ef96c62d76a7a3cd1b24e5c7ecc9100c9d6fd87e828b820594eeee437d6" - Structure needs cleaning (os error 117)
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 601 failed - backup write data failed: command error: write_data upload error: pipelined request failed: inserting chunk on store 'datastorecasta' failed for 072a0ef96c62d76a7a3cd1b24e5c7ecc9100c9d6fd87e828b820594eeee437d6 - Atomic rename failed for file "/mnt/datastore/datastorecasta/.chunks/072a/072a0ef96c62d76a7a3cd1b24e5c7ecc9100c9d6fd87e828b820594eeee437d6" - Structure needs cleaning (os error 117)
INFO: Failed at 2023-09-08 03:17:02
INFO: Backup job finished with errors
TASK ERROR: job errors
