I’m currently running Proxmox with an Intel Core i9-10885H (16 cores @ 2.4 GHz) and 128 GB of RAM. My setup includes NVMe drives for both the Proxmox OS and backups.
I’ve noticed that during VM backups, the average transfer speed hovers around 50 MiB/s, which seems lower than expected given the hardware capabilities. Here are some details from my recent backup:
Thanks in advance!
I’ve noticed that during VM backups, the average transfer speed hovers around 50 MiB/s, which seems lower than expected given the hardware capabilities. Here are some details from my recent backup:
- Backup Size: 32.00 GiB
- Time Taken: 652 seconds (10 minutes 52 seconds)
- Average Transfer Speed: 50.3 MiB/s
- Compressed Archive Size: 2.62 GiB
- Sparse Data: 89% total zero data
- Using vzdump with snapshot mode and ZSTD compression.
- What configurations or optimizations can I implement in ZFS to improve backup speeds?
- Are there specific I/O scheduler settings you would recommend for a setup like mine?
- Could the compression settings be affecting performance, and how can I best configure them?
Thanks in advance!
Code:
root@pmx:~# proxmox-backup-client benchmark
SHA256 speed: 535.04 MB/s
Compression speed: 583.60 MB/s
Decompress speed: 943.24 MB/s
AES256/GCM speed: 1879.14 MB/s
Verify speed: 334.60 MB/s
┌───────────────────────────────────┬────────────────────┐
│ Name │ Value │
╞═══════════════════════════════════╪════════════════════╡
│ TLS (maximal backup upload speed) │ not tested │
├───────────────────────────────────┼────────────────────┤
│ SHA256 checksum computation speed │ 535.04 MB/s (26%) │
├───────────────────────────────────┼────────────────────┤
│ ZStd level 1 compression speed │ 583.60 MB/s (78%) │
├───────────────────────────────────┼────────────────────┤
│ ZStd level 1 decompression speed │ 943.24 MB/s (79%) │
├───────────────────────────────────┼────────────────────┤
│ Chunk verification speed │ 334.60 MB/s (44%) │
├───────────────────────────────────┼────────────────────┤
│ AES256 GCM encryption speed │ 1879.14 MB/s (52%) │
└───────────────────────────────────┴────────────────────┘
root@pmx:~#
Code:
()
INFO: starting new backup job: vzdump 9998 --notification-mode auto --storage nvme --mode snapshot --notes-template '{{guestname}}' --node pmx --compress zstd --remove 0
INFO: Starting Backup of VM 9998 (qemu)
INFO: Backup started at 2024-09-22 14:50:48
INFO: status = running
INFO: VM Name: FW
INFO: include disk 'scsi0' 'local-lvm:vm-9998-disk-0' 32G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating vzdump archive '/nvme/dump/vzdump-qemu-9998-2024_09_22-14_50_48.vma.zst'
INFO: started backup task 'f2443f9a-ccfb-4861-bb7b-23215090ec70'
INFO: resuming VM again
INFO: 0% (118.2 MiB of 32.0 GiB) in 3s, read: 39.4 MiB/s, write: 30.1 MiB/s
INFO: 1% (351.9 MiB of 32.0 GiB) in 9s, read: 38.9 MiB/s, write: 38.9 MiB/s
INFO: 2% (862.9 MiB of 32.0 GiB) in 13s, read: 127.8 MiB/s, write: 30.7 MiB/s
INFO: 5% (1.6 GiB of 32.0 GiB) in 16s, read: 271.0 MiB/s, write: 31.7 MiB/s
INFO: 10% (3.3 GiB of 32.0 GiB) in 19s, read: 557.0 MiB/s, write: 18.1 MiB/s
INFO: 11% (3.5 GiB of 32.0 GiB) in 26s, read: 39.2 MiB/s, write: 29.1 MiB/s
INFO: 12% (3.8 GiB of 32.0 GiB) in 34s, read: 39.2 MiB/s, write: 30.0 MiB/s
INFO: 13% (4.2 GiB of 32.0 GiB) in 43s, read: 39.7 MiB/s, write: 22.4 MiB/s
INFO: 14% (4.5 GiB of 32.0 GiB) in 51s, read: 39.8 MiB/s, write: 32.6 MiB/s
INFO: 15% (4.8 GiB of 32.0 GiB) in 59s, read: 39.8 MiB/s, write: 300.0 KiB/s
INFO: 16% (5.1 GiB of 32.0 GiB) in 1m 7s, read: 40.5 MiB/s, write: 16.0 KiB/s
INFO: 17% (5.4 GiB of 32.0 GiB) in 1m 15s, read: 40.3 MiB/s, write: 8.5 KiB/s
INFO: 18% (5.8 GiB of 32.0 GiB) in 1m 24s, read: 39.7 MiB/s, write: 96.9 KiB/s
INFO: 19% (6.1 GiB of 32.0 GiB) in 1m 32s, read: 38.9 MiB/s, write: 374.0 KiB/s
INFO: 20% (6.4 GiB of 32.0 GiB) in 1m 40s, read: 39.5 MiB/s, write: 50.5 KiB/s
INFO: 30% (9.8 GiB of 32.0 GiB) in 1m 43s, read: 1.1 GiB/s, write: 11.1 MiB/s
INFO: 31% (10.0 GiB of 32.0 GiB) in 1m 46s, read: 39.3 MiB/s, write: 7.3 MiB/s
INFO: 32% (10.3 GiB of 32.0 GiB) in 1m 54s, read: 38.5 MiB/s, write: 37.7 MiB/s
INFO: 33% (10.6 GiB of 32.0 GiB) in 2m 2s, read: 39.9 MiB/s, write: 38.6 MiB/s
INFO: 34% (10.9 GiB of 32.0 GiB) in 2m 10s, read: 41.0 MiB/s, write: 40.0 MiB/s
INFO: 35% (11.2 GiB of 32.0 GiB) in 2m 18s, read: 40.8 MiB/s, write: 36.2 MiB/s
INFO: 36% (11.5 GiB of 32.0 GiB) in 2m 26s, read: 41.4 MiB/s, write: 16.0 KiB/s
INFO: 37% (11.9 GiB of 32.0 GiB) in 2m 34s, read: 41.6 MiB/s, write: 13.5 KiB/s
INFO: 38% (12.2 GiB of 32.0 GiB) in 2m 42s, read: 41.1 MiB/s, write: 41.0 KiB/s
INFO: 39% (12.5 GiB of 32.0 GiB) in 2m 50s, read: 41.4 MiB/s, write: 0 B/s
INFO: 40% (12.8 GiB of 32.0 GiB) in 2m 58s, read: 41.5 MiB/s, write: 0 B/s
INFO: 41% (13.1 GiB of 32.0 GiB) in 3m 6s, read: 41.4 MiB/s, write: 2.0 KiB/s
INFO: 42% (13.5 GiB of 32.0 GiB) in 3m 14s, read: 41.6 MiB/s, write: 2.0 KiB/s
INFO: 43% (13.8 GiB of 32.0 GiB) in 3m 21s, read: 41.9 MiB/s, write: 9.1 KiB/s
INFO: 44% (14.1 GiB of 32.0 GiB) in 3m 29s, read: 41.5 MiB/s, write: 2.1 MiB/s
INFO: 45% (14.4 GiB of 32.0 GiB) in 3m 37s, read: 41.4 MiB/s, write: 0 B/s
INFO: 46% (14.7 GiB of 32.0 GiB) in 3m 45s, read: 41.6 MiB/s, write: 0 B/s
INFO: 47% (15.1 GiB of 32.0 GiB) in 3m 53s, read: 41.8 MiB/s, write: 0 B/s
INFO: 48% (15.4 GiB of 32.0 GiB) in 4m 1s, read: 41.8 MiB/s, write: 0 B/s
INFO: 49% (15.7 GiB of 32.0 GiB) in 4m 9s, read: 41.8 MiB/s, write: 0 B/s
INFO: 50% (16.0 GiB of 32.0 GiB) in 4m 17s, read: 40.7 MiB/s, write: 9.6 MiB/s
INFO: 51% (16.3 GiB of 32.0 GiB) in 4m 25s, read: 40.3 MiB/s, write: 36.5 MiB/s
INFO: 52% (16.7 GiB of 32.0 GiB) in 4m 33s, read: 41.2 MiB/s, write: 201.5 KiB/s
INFO: 53% (17.0 GiB of 32.0 GiB) in 4m 41s, read: 40.9 MiB/s, write: 15.0 KiB/s
INFO: 54% (17.3 GiB of 32.0 GiB) in 4m 49s, read: 41.1 MiB/s, write: 43.0 KiB/s
INFO: 55% (17.6 GiB of 32.0 GiB) in 4m 57s, read: 41.1 MiB/s, write: 13.0 KiB/s
INFO: 56% (17.9 GiB of 32.0 GiB) in 5m 5s, read: 40.5 MiB/s, write: 2.1 MiB/s
INFO: 57% (18.3 GiB of 32.0 GiB) in 5m 13s, read: 41.8 MiB/s, write: 4.5 MiB/s
INFO: 58% (18.6 GiB of 32.0 GiB) in 5m 21s, read: 41.5 MiB/s, write: 8.0 KiB/s
INFO: 59% (18.9 GiB of 32.0 GiB) in 5m 28s, read: 41.6 MiB/s, write: 0 B/s
INFO: 60% (19.2 GiB of 32.0 GiB) in 5m 36s, read: 41.8 MiB/s, write: 0 B/s
INFO: 61% (19.5 GiB of 32.0 GiB) in 5m 44s, read: 41.9 MiB/s, write: 0 B/s
INFO: 62% (19.9 GiB of 32.0 GiB) in 5m 52s, read: 41.5 MiB/s, write: 10.5 KiB/s
INFO: 63% (20.2 GiB of 32.0 GiB) in 6m, read: 41.7 MiB/s, write: 12.3 MiB/s
INFO: 64% (20.5 GiB of 32.0 GiB) in 6m 8s, read: 41.8 MiB/s, write: 0 B/s
INFO: 65% (20.8 GiB of 32.0 GiB) in 6m 16s, read: 41.6 MiB/s, write: 0 B/s
INFO: 66% (21.2 GiB of 32.0 GiB) in 6m 24s, read: 41.4 MiB/s, write: 16.0 KiB/s
INFO: 67% (21.4 GiB of 32.0 GiB) in 6m 31s, read: 41.7 MiB/s, write: 18.3 KiB/s
INFO: 68% (21.8 GiB of 32.0 GiB) in 6m 39s, read: 41.0 MiB/s, write: 8.0 KiB/s
INFO: 69% (22.1 GiB of 32.0 GiB) in 6m 47s, read: 41.7 MiB/s, write: 297.0 KiB/s
INFO: 70% (22.4 GiB of 32.0 GiB) in 6m 55s, read: 42.0 MiB/s, write: 0 B/s
INFO: 71% (22.7 GiB of 32.0 GiB) in 7m 3s, read: 42.0 MiB/s, write: 0 B/s
INFO: 72% (23.1 GiB of 32.0 GiB) in 7m 11s, read: 41.9 MiB/s, write: 7.0 KiB/s
INFO: 73% (23.4 GiB of 32.0 GiB) in 7m 19s, read: 41.9 MiB/s, write: 0 B/s
INFO: 74% (23.7 GiB of 32.0 GiB) in 7m 26s, read: 41.9 MiB/s, write: 18.3 KiB/s
INFO: 75% (24.0 GiB of 32.0 GiB) in 7m 34s, read: 42.0 MiB/s, write: 2.1 MiB/s
INFO: 76% (24.3 GiB of 32.0 GiB) in 7m 42s, read: 41.9 MiB/s, write: 0 B/s
INFO: 77% (24.7 GiB of 32.0 GiB) in 7m 50s, read: 41.4 MiB/s, write: 0 B/s
INFO: 78% (25.0 GiB of 32.0 GiB) in 7m 58s, read: 41.6 MiB/s, write: 0 B/s
INFO: 79% (25.3 GiB of 32.0 GiB) in 8m 6s, read: 41.8 MiB/s, write: 0 B/s
INFO: 80% (25.6 GiB of 32.0 GiB) in 8m 14s, read: 41.2 MiB/s, write: 2.0 KiB/s
INFO: 81% (25.9 GiB of 32.0 GiB) in 8m 21s, read: 41.4 MiB/s, write: 24.6 KiB/s
INFO: 82% (26.2 GiB of 32.0 GiB) in 8m 29s, read: 41.6 MiB/s, write: 0 B/s
INFO: 83% (26.6 GiB of 32.0 GiB) in 8m 37s, read: 41.6 MiB/s, write: 8.0 KiB/s
INFO: 84% (26.9 GiB of 32.0 GiB) in 8m 45s, read: 41.4 MiB/s, write: 0 B/s
INFO: 85% (27.2 GiB of 32.0 GiB) in 8m 53s, read: 41.5 MiB/s, write: 0 B/s
INFO: 86% (27.5 GiB of 32.0 GiB) in 9m 1s, read: 41.1 MiB/s, write: 0 B/s
INFO: 87% (27.9 GiB of 32.0 GiB) in 9m 10s, read: 38.0 MiB/s, write: 121.8 KiB/s
INFO: 88% (28.2 GiB of 32.0 GiB) in 9m 18s, read: 41.5 MiB/s, write: 1.9 MiB/s
INFO: 89% (28.5 GiB of 32.0 GiB) in 9m 25s, read: 41.3 MiB/s, write: 0 B/s
INFO: 90% (28.8 GiB of 32.0 GiB) in 9m 33s, read: 41.2 MiB/s, write: 0 B/s
INFO: 91% (29.1 GiB of 32.0 GiB) in 9m 41s, read: 41.6 MiB/s, write: 0 B/s
INFO: 92% (29.5 GiB of 32.0 GiB) in 9m 49s, read: 41.6 MiB/s, write: 0 B/s
INFO: 93% (29.8 GiB of 32.0 GiB) in 9m 57s, read: 42.0 MiB/s, write: 4.0 KiB/s
INFO: 94% (30.1 GiB of 32.0 GiB) in 10m 5s, read: 41.5 MiB/s, write: 141.5 KiB/s
INFO: 95% (30.4 GiB of 32.0 GiB) in 10m 13s, read: 41.9 MiB/s, write: 0 B/s
INFO: 96% (30.8 GiB of 32.0 GiB) in 10m 21s, read: 41.6 MiB/s, write: 0 B/s
INFO: 97% (31.0 GiB of 32.0 GiB) in 10m 28s, read: 41.1 MiB/s, write: 0 B/s
INFO: 98% (31.4 GiB of 32.0 GiB) in 10m 36s, read: 41.8 MiB/s, write: 0 B/s
INFO: 99% (31.7 GiB of 32.0 GiB) in 10m 44s, read: 41.6 MiB/s, write: 0 B/s
INFO: 100% (32.0 GiB of 32.0 GiB) in 10m 52s, read: 39.4 MiB/s, write: 2.0 KiB/s
INFO: backup is sparse: 28.72 GiB (89%) total zero data
INFO: transferred 32.00 GiB in 652 seconds (50.3 MiB/s)
INFO: archive file size: 2.62GB
INFO: adding notes to backup
INFO: Finished Backup of VM 9998 (00:10:52)
INFO: Backup finished at 2024-09-22 15:01:40
INFO: Backup job finished successfully
TASK OK