I have moved this thread from the Proxmox Backup forum as the issue does not involve a Proxmox Backup server.
I am attempting to back up a VM with an 18TB disk allocated of which 17.6TiB is used to a 20TB external HDD. In the local-lvm "vm-201-disk-1" is listed as having a size of 19.33TB Unfortunately the backup has now failed twice with the same error "zstd: error 70 : Write error : cannot write block : File too large" followed by the error "ERROR: vma_queue_write: write error - Broken pipe". I have posted the full log below. I am gathering that this error is due to a lack of space but due to the time this backup takes I have not been able to see the external disk usage at the moment the backups fails, but the disk was only shown to be 85% full (of 19.93TB) at the time the backup was 93% complete. I believe I need to delete some data to make the backup fit but I can not seem to find any documentation that indicates a ratio or formula to calculate how much larger a backup volume needs to be than the size of the backup I am trying to take as I would think that over 2TiB would be enough space for a compression temp directory. Does anyone have experience with this issue, know where I can find more verbose logs, or know of some piece of documentation I am missing? Any insight or suggestions would be very much appreciated.
I am attempting to back up a VM with an 18TB disk allocated of which 17.6TiB is used to a 20TB external HDD. In the local-lvm "vm-201-disk-1" is listed as having a size of 19.33TB Unfortunately the backup has now failed twice with the same error "zstd: error 70 : Write error : cannot write block : File too large" followed by the error "ERROR: vma_queue_write: write error - Broken pipe". I have posted the full log below. I am gathering that this error is due to a lack of space but due to the time this backup takes I have not been able to see the external disk usage at the moment the backups fails, but the disk was only shown to be 85% full (of 19.93TB) at the time the backup was 93% complete. I believe I need to delete some data to make the backup fit but I can not seem to find any documentation that indicates a ratio or formula to calculate how much larger a backup volume needs to be than the size of the backup I am trying to take as I would think that over 2TiB would be enough space for a compression temp directory. Does anyone have experience with this issue, know where I can find more verbose logs, or know of some piece of documentation I am missing? Any insight or suggestions would be very much appreciated.
Code:
INFO: starting new backup job: vzdump 201 --compress zstd --notification-mode auto --remove 0 --node proxmox1 --mode stop --storage [REMOVED]Backup --notes-template '{{guestname}}'
INFO: Starting Backup of VM 201 (qemu)
INFO: Backup started at 2024-08-28 18:26:42
INFO: status = running
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: VM Name: [REMOVED]
INFO: include disk 'scsi0' 'local-lvm:vm-201-disk-1' 18000G
INFO: include disk 'efidisk0' 'local-lvm:vm-201-disk-0' 4M
INFO: stopping virtual guest
INFO: snapshots found (not included into backup)
INFO: creating vzdump archive '/media/usb-drive/dump/vzdump-qemu-201-2024_08_28-18_26_42.vma.zst'
INFO: starting kvm to execute backup task
INFO: started backup task '698251e8-5f25-4e57-b424-92251a0bf521'
INFO: resuming VM again after 53 seconds
INFO: 0% (1.8 GiB of 17.6 TiB) in 3s, read: 601.8 MiB/s, write: 195.4 MiB/s
INFO: 1% (180.0 GiB of 17.6 TiB) in 1h 16m 6s, read: 40.0 MiB/s, write: 39.3 MiB/s
INFO: 2% (360.0 GiB of 17.6 TiB) in 2h 35m 52s, read: 38.5 MiB/s, write: 37.9 MiB/s
INFO: 3% (540.0 GiB of 17.6 TiB) in 3h 56m 25s, read: 38.1 MiB/s, write: 37.5 MiB/s
INFO: 4% (720.0 GiB of 17.6 TiB) in 5h 15m 45s, read: 38.7 MiB/s, write: 37.7 MiB/s
INFO: 5% (900.0 GiB of 17.6 TiB) in 6h 35m 48s, read: 38.4 MiB/s, write: 37.7 MiB/s
INFO: 6% (1.1 TiB of 17.6 TiB) in 7h 56m 39s, read: 38.0 MiB/s, write: 37.4 MiB/s
INFO: 7% (1.2 TiB of 17.6 TiB) in 9h 16m 18s, read: 38.6 MiB/s, write: 38.0 MiB/s
INFO: 8% (1.4 TiB of 17.6 TiB) in 10h 38m 4s, read: 37.6 MiB/s, write: 37.0 MiB/s
INFO: 9% (1.6 TiB of 17.6 TiB) in 11h 59m 17s, read: 37.8 MiB/s, write: 37.2 MiB/s
INFO: 10% (1.8 TiB of 17.6 TiB) in 13h 18m 49s, read: 38.6 MiB/s, write: 37.4 MiB/s
INFO: 11% (1.9 TiB of 17.6 TiB) in 14h 38m 30s, read: 38.5 MiB/s, write: 37.5 MiB/s
INFO: 12% (2.1 TiB of 17.6 TiB) in 15h 55m 51s, read: 39.7 MiB/s, write: 37.7 MiB/s
INFO: 13% (2.3 TiB of 17.6 TiB) in 17h 16m 10s, read: 38.2 MiB/s, write: 37.4 MiB/s
INFO: 14% (2.5 TiB of 17.6 TiB) in 18h 35m 19s, read: 38.8 MiB/s, write: 37.3 MiB/s
INFO: 15% (2.6 TiB of 17.6 TiB) in 19h 54m 10s, read: 39.0 MiB/s, write: 37.6 MiB/s
INFO: 16% (2.8 TiB of 17.6 TiB) in 21h 14m 38s, read: 38.2 MiB/s, write: 37.5 MiB/s
INFO: 17% (3.0 TiB of 17.6 TiB) in 22h 33m 44s, read: 38.8 MiB/s, write: 37.6 MiB/s
INFO: 18% (3.2 TiB of 17.6 TiB) in 23h 51m 33s, read: 39.5 MiB/s, write: 37.2 MiB/s
INFO: 19% (3.3 TiB of 17.6 TiB) in 1d 1h 10m 15s, read: 39.0 MiB/s, write: 37.3 MiB/s
INFO: 20% (3.5 TiB of 17.6 TiB) in 1d 2h 30m 22s, read: 38.3 MiB/s, write: 37.3 MiB/s
INFO: 21% (3.7 TiB of 17.6 TiB) in 1d 3h 46m 39s, read: 40.3 MiB/s, write: 37.4 MiB/s
INFO: 22% (3.9 TiB of 17.6 TiB) in 1d 5h 3m 23s, read: 40.0 MiB/s, write: 37.6 MiB/s
INFO: 23% (4.0 TiB of 17.6 TiB) in 1d 6h 19m 43s, read: 40.2 MiB/s, write: 37.9 MiB/s
INFO: 24% (4.2 TiB of 17.6 TiB) in 1d 7h 36m 40s, read: 39.9 MiB/s, write: 37.6 MiB/s
INFO: 25% (4.4 TiB of 17.6 TiB) in 1d 8h 56m 58s, read: 38.3 MiB/s, write: 37.0 MiB/s
INFO: 26% (4.6 TiB of 17.6 TiB) in 1d 10h 11m 6s, read: 41.4 MiB/s, write: 36.9 MiB/s
INFO: 27% (4.7 TiB of 17.6 TiB) in 1d 11h 30m 26s, read: 38.7 MiB/s, write: 36.9 MiB/s
INFO: 28% (4.9 TiB of 17.6 TiB) in 1d 12h 50m 41s, read: 38.3 MiB/s, write: 36.7 MiB/s
INFO: 29% (5.1 TiB of 17.6 TiB) in 1d 14h 7m 38s, read: 39.9 MiB/s, write: 37.7 MiB/s
INFO: 30% (5.3 TiB of 17.6 TiB) in 1d 15h 26m 10s, read: 39.1 MiB/s, write: 38.1 MiB/s
INFO: 31% (5.4 TiB of 17.6 TiB) in 1d 16h 41m 28s, read: 40.8 MiB/s, write: 38.3 MiB/s
INFO: 32% (5.6 TiB of 17.6 TiB) in 1d 17h 59m 18s, read: 39.5 MiB/s, write: 37.6 MiB/s
INFO: 33% (5.8 TiB of 17.6 TiB) in 1d 19h 15m 55s, read: 40.1 MiB/s, write: 37.8 MiB/s
INFO: 34% (6.0 TiB of 17.6 TiB) in 1d 20h 31m 54s, read: 40.4 MiB/s, write: 38.1 MiB/s
INFO: 35% (6.2 TiB of 17.6 TiB) in 1d 21h 51m 22s, read: 38.7 MiB/s, write: 37.4 MiB/s
INFO: 36% (6.3 TiB of 17.6 TiB) in 1d 23h 8m 8s, read: 40.0 MiB/s, write: 37.9 MiB/s
INFO: 37% (6.5 TiB of 17.6 TiB) in 2d 26m 17s, read: 39.3 MiB/s, write: 37.4 MiB/s
INFO: 38% (6.7 TiB of 17.6 TiB) in 2d 1h 33m 2s, read: 46.0 MiB/s, write: 37.4 MiB/s
INFO: 39% (6.9 TiB of 17.6 TiB) in 2d 2h 52m 42s, read: 38.6 MiB/s, write: 36.9 MiB/s
INFO: 40% (7.0 TiB of 17.6 TiB) in 2d 4h 11m 47s, read: 38.8 MiB/s, write: 36.9 MiB/s
INFO: 41% (7.2 TiB of 17.6 TiB) in 2d 5h 32m 42s, read: 38.0 MiB/s, write: 36.9 MiB/s
INFO: 42% (7.4 TiB of 17.6 TiB) in 2d 6h 54m 25s, read: 37.6 MiB/s, write: 36.7 MiB/s
INFO: 43% (7.6 TiB of 17.6 TiB) in 2d 8h 15m 23s, read: 37.9 MiB/s, write: 37.2 MiB/s
INFO: 44% (7.7 TiB of 17.6 TiB) in 2d 9h 35m 51s, read: 38.2 MiB/s, write: 37.0 MiB/s
INFO: 45% (7.9 TiB of 17.6 TiB) in 2d 10h 57m 44s, read: 37.5 MiB/s, write: 36.8 MiB/s
INFO: 46% (8.1 TiB of 17.6 TiB) in 2d 12h 19m 46s, read: 37.4 MiB/s, write: 36.8 MiB/s
INFO: 47% (8.3 TiB of 17.6 TiB) in 2d 13h 41m 46s, read: 37.5 MiB/s, write: 36.7 MiB/s
INFO: 48% (8.4 TiB of 17.6 TiB) in 2d 15h 59s, read: 38.8 MiB/s, write: 36.7 MiB/s
INFO: 49% (8.6 TiB of 17.6 TiB) in 2d 16h 20m 3s, read: 38.9 MiB/s, write: 36.8 MiB/s
INFO: 50% (8.8 TiB of 17.6 TiB) in 2d 17h 41m 51s, read: 37.6 MiB/s, write: 36.8 MiB/s
INFO: 51% (9.0 TiB of 17.6 TiB) in 2d 19h 42s, read: 39.0 MiB/s, write: 36.8 MiB/s
INFO: 52% (9.1 TiB of 17.6 TiB) in 2d 20h 18m 22s, read: 39.6 MiB/s, write: 37.2 MiB/s
INFO: 53% (9.3 TiB of 17.6 TiB) in 2d 21h 36m 53s, read: 39.1 MiB/s, write: 36.8 MiB/s
INFO: 54% (9.5 TiB of 17.6 TiB) in 2d 22h 55m 15s, read: 39.2 MiB/s, write: 36.9 MiB/s
INFO: 55% (9.7 TiB of 17.6 TiB) in 3d 14m 50s, read: 38.6 MiB/s, write: 36.9 MiB/s
INFO: 56% (9.8 TiB of 17.6 TiB) in 3d 1h 34m 27s, read: 38.6 MiB/s, write: 37.0 MiB/s
INFO: 57% (10.0 TiB of 17.6 TiB) in 3d 2h 56m 19s, read: 37.5 MiB/s, write: 36.8 MiB/s
INFO: 58% (10.2 TiB of 17.6 TiB) in 3d 4h 16m 56s, read: 38.1 MiB/s, write: 37.4 MiB/s
INFO: 59% (10.4 TiB of 17.6 TiB) in 3d 5h 36m 21s, read: 38.7 MiB/s, write: 38.0 MiB/s
INFO: 60% (10.5 TiB of 17.6 TiB) in 3d 6h 57m 36s, read: 37.8 MiB/s, write: 37.2 MiB/s
INFO: 61% (10.7 TiB of 17.6 TiB) in 3d 8h 19m 32s, read: 37.5 MiB/s, write: 36.8 MiB/s
INFO: 62% (10.9 TiB of 17.6 TiB) in 3d 9h 41m 30s, read: 37.5 MiB/s, write: 36.8 MiB/s
INFO: 63% (11.1 TiB of 17.6 TiB) in 3d 11h 3m 39s, read: 37.4 MiB/s, write: 36.7 MiB/s
INFO: 64% (11.3 TiB of 17.6 TiB) in 3d 12h 25m 29s, read: 37.5 MiB/s, write: 36.9 MiB/s
INFO: 65% (11.4 TiB of 17.6 TiB) in 3d 13h 47m 12s, read: 37.6 MiB/s, write: 36.9 MiB/s
INFO: 66% (11.6 TiB of 17.6 TiB) in 3d 15h 9m 16s, read: 37.4 MiB/s, write: 36.8 MiB/s
INFO: 67% (11.8 TiB of 17.6 TiB) in 3d 16h 28m 12s, read: 38.9 MiB/s, write: 38.2 MiB/s
INFO: 68% (12.0 TiB of 17.6 TiB) in 3d 17h 49m 51s, read: 37.6 MiB/s, write: 36.9 MiB/s
INFO: 69% (12.1 TiB of 17.6 TiB) in 3d 19h 9m 9s, read: 38.7 MiB/s, write: 36.7 MiB/s
INFO: 70% (12.3 TiB of 17.6 TiB) in 3d 20h 31m 30s, read: 37.3 MiB/s, write: 36.6 MiB/s
INFO: 71% (12.5 TiB of 17.6 TiB) in 3d 21h 53m 56s, read: 37.3 MiB/s, write: 36.6 MiB/s
INFO: 72% (12.7 TiB of 17.6 TiB) in 3d 23h 16m 21s, read: 37.3 MiB/s, write: 36.6 MiB/s
INFO: 73% (12.8 TiB of 17.6 TiB) in 4d 38m 32s, read: 37.4 MiB/s, write: 36.7 MiB/s
INFO: 74% (13.0 TiB of 17.6 TiB) in 4d 2h 36s, read: 37.4 MiB/s, write: 36.8 MiB/s
INFO: 75% (13.2 TiB of 17.6 TiB) in 4d 3h 22m 19s, read: 37.6 MiB/s, write: 36.9 MiB/s
INFO: 76% (13.4 TiB of 17.6 TiB) in 4d 4h 44m 35s, read: 37.3 MiB/s, write: 36.7 MiB/s
INFO: 77% (13.5 TiB of 17.6 TiB) in 4d 6h 6m 9s, read: 37.7 MiB/s, write: 37.0 MiB/s
INFO: 78% (13.7 TiB of 17.6 TiB) in 4d 7h 27m 11s, read: 37.9 MiB/s, write: 37.2 MiB/s
INFO: 79% (13.9 TiB of 17.6 TiB) in 4d 8h 49m 7s, read: 37.5 MiB/s, write: 36.8 MiB/s
INFO: 80% (14.1 TiB of 17.6 TiB) in 4d 10h 10m 56s, read: 37.5 MiB/s, write: 36.9 MiB/s
INFO: 81% (14.2 TiB of 17.6 TiB) in 4d 11h 32m 36s, read: 37.6 MiB/s, write: 36.9 MiB/s
INFO: 82% (14.4 TiB of 17.6 TiB) in 4d 12h 54m 33s, read: 37.5 MiB/s, write: 36.8 MiB/s
INFO: 83% (14.6 TiB of 17.6 TiB) in 4d 14h 16m 41s, read: 37.4 MiB/s, write: 36.7 MiB/s
INFO: 84% (14.8 TiB of 17.6 TiB) in 4d 15h 38m 41s, read: 37.5 MiB/s, write: 36.7 MiB/s
INFO: 85% (14.9 TiB of 17.6 TiB) in 4d 17h 37s, read: 37.5 MiB/s, write: 36.8 MiB/s
INFO: 86% (15.1 TiB of 17.6 TiB) in 4d 18h 22m 31s, read: 37.5 MiB/s, write: 36.8 MiB/s
INFO: 87% (15.3 TiB of 17.6 TiB) in 4d 19h 44m 39s, read: 37.4 MiB/s, write: 36.7 MiB/s
INFO: 88% (15.5 TiB of 17.6 TiB) in 4d 21h 5m 14s, read: 38.1 MiB/s, write: 37.4 MiB/s
INFO: 89% (15.6 TiB of 17.6 TiB) in 4d 22h 27m 1s, read: 37.6 MiB/s, write: 36.8 MiB/s
INFO: 90% (15.8 TiB of 17.6 TiB) in 4d 23h 45m 42s, read: 39.0 MiB/s, write: 37.1 MiB/s
INFO: 91% (16.0 TiB of 17.6 TiB) in 5d 1h 3m 24s, read: 39.6 MiB/s, write: 37.2 MiB/s
INFO: 92% (16.2 TiB of 17.6 TiB) in 5d 2h 21m 38s, read: 39.2 MiB/s, write: 36.9 MiB/s
INFO: 93% (16.3 TiB of 17.6 TiB) in 5d 3h 39m 30s, read: 39.5 MiB/s, write: 37.1 MiB/s
INFO: 94% (16.5 TiB of 17.6 TiB) in 5d 4h 55m 23s, read: 40.5 MiB/s, write: 37.2 MiB/s
INFO: 95% (16.7 TiB of 17.6 TiB) in 5d 5h 59m 34s, read: 47.9 MiB/s, write: 38.8 MiB/s
INFO: 96% (16.9 TiB of 17.6 TiB) in 5d 6h 31m 27s, read: 96.4 MiB/s, write: 42.8 MiB/s
INFO: 97% (17.1 TiB of 17.6 TiB) in 5d 7h 11m 38s, read: 76.4 MiB/s, write: 37.7 MiB/s
zstd: error 70 : Write error : cannot write block : File too large
INFO: 97% (17.1 TiB of 17.6 TiB) in 5d 7h 17m 48s, read: 40.9 MiB/s, write: 38.3 MiB/s
ERROR: vma_queue_write: write error - Broken pipe
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 201 failed - vma_queue_write: write error - Broken pipe
INFO: Failed at 2024-09-03 01:46:19
INFO: Backup job finished with errors
INFO: notified via target `mail-to-root`
TASK ERROR: job errors
Last edited:
