Hello,
I have been struggling with PBS incremental backup, from what i have learned and investigated the forced zstd compression is done by the PVE.
Yes my PVE may be not that powerful
/usr/bin/proxmox-backup-client benchmark --repository backup@pbs@10.1.4.20
BS-STOR01
Uploaded 356 chunks in 5 seconds.
Time per request: 14101 microseconds.
TLS speed: 297.44 MB/s
SHA256 speed: 366.61 MB/s
Compression speed: 335.87 MB/s
Decompress speed: 509.30 MB/s
AES256/GCM speed: 940.45 MB/s
Verify speed: 204.20 MB/s
┌───────────────────────────────────┬───────────────────┐
│ Name │ Value │
╞═══════════════════════════════════╪═══════════════════╡
│ TLS (maximal backup upload speed) │ 297.44 MB/s (24%) │
├───────────────────────────────────┼───────────────────┤
│ SHA256 checksum computation speed │ 366.61 MB/s (18%) │
├───────────────────────────────────┼───────────────────┤
│ ZStd level 1 compression speed │ 335.87 MB/s (45%) │
├───────────────────────────────────┼───────────────────┤
│ ZStd level 1 decompression speed │ 509.30 MB/s (43%) │
├───────────────────────────────────┼───────────────────┤
│ Chunk verification speed │ 204.20 MB/s (27%) │
├───────────────────────────────────┼───────────────────┤
│ AES256 GCM encryption speed │ 940.45 MB/s (26%) │
└───────────────────────────────────┴───────────────────┘
So this limits my upoload to 200 MB/s when testing the same with no compression backup will reach 990 MB/s , but it should be possible to use multiple cores while compressing zstd or compile proxmox-backup-client to use multiple cores to cope for that lack of speed of the CPU, because i have tested that when sendig to PBS it uses a single core.
Or the other solution will be to give the user the option not to compress the backup, this because we can use compression on our storage and saving space.
Thanks for your help
I have been struggling with PBS incremental backup, from what i have learned and investigated the forced zstd compression is done by the PVE.
Yes my PVE may be not that powerful
/usr/bin/proxmox-backup-client benchmark --repository backup@pbs@10.1.4.20
BS-STOR01Uploaded 356 chunks in 5 seconds.
Time per request: 14101 microseconds.
TLS speed: 297.44 MB/s
SHA256 speed: 366.61 MB/s
Compression speed: 335.87 MB/s
Decompress speed: 509.30 MB/s
AES256/GCM speed: 940.45 MB/s
Verify speed: 204.20 MB/s
┌───────────────────────────────────┬───────────────────┐
│ Name │ Value │
╞═══════════════════════════════════╪═══════════════════╡
│ TLS (maximal backup upload speed) │ 297.44 MB/s (24%) │
├───────────────────────────────────┼───────────────────┤
│ SHA256 checksum computation speed │ 366.61 MB/s (18%) │
├───────────────────────────────────┼───────────────────┤
│ ZStd level 1 compression speed │ 335.87 MB/s (45%) │
├───────────────────────────────────┼───────────────────┤
│ ZStd level 1 decompression speed │ 509.30 MB/s (43%) │
├───────────────────────────────────┼───────────────────┤
│ Chunk verification speed │ 204.20 MB/s (27%) │
├───────────────────────────────────┼───────────────────┤
│ AES256 GCM encryption speed │ 940.45 MB/s (26%) │
└───────────────────────────────────┴───────────────────┘
So this limits my upoload to 200 MB/s when testing the same with no compression backup will reach 990 MB/s , but it should be possible to use multiple cores while compressing zstd or compile proxmox-backup-client to use multiple cores to cope for that lack of speed of the CPU, because i have tested that when sendig to PBS it uses a single core.
Or the other solution will be to give the user the option not to compress the backup, this because we can use compression on our storage and saving space.
Thanks for your help