slow zstd compression speeds

[Kodey]

Most of the benchmarks I've seen here have a much higher zstd de/compression speeds.
I'm trying to understand why mine look so low.
What are some areas to look at that could help my performance?
Storage is USB 3.2 drive running xfs. Setup is pbs and pve running in place

proxmox-backup-client benchmark --repository PBS-C-WD14TB
Uploaded 2014 chunks in 5 seconds.
Time per request: 2484 microseconds.
TLS speed: 1688.34 MB/s
SHA256 speed: 2316.06 MB/s
Compression speed: 890.29 MB/s
Decompress speed: 1151.49 MB/s
AES256/GCM speed: 2705.21 MB/s
Verify speed: 770.13 MB/s
┌───────────────────────────────────┬─────────────────────┐
│ Name                              │ Value               │
╞═══════════════════════════════════╪═════════════════════╡
│ TLS (maximal backup upload speed) │ 1688.34 MB/s (137%) │
├───────────────────────────────────┼─────────────────────┤
│ SHA256 checksum computation speed │ 2316.06 MB/s (115%) │
├───────────────────────────────────┼─────────────────────┤
│ ZStd level 1 compression speed    │ 890.29 MB/s (118%)  │
├───────────────────────────────────┼─────────────────────┤
│ ZStd level 1 decompression speed  │ 1151.49 MB/s (96%)  │
├───────────────────────────────────┼─────────────────────┤
│ Chunk verification speed          │ 770.13 MB/s (102%)  │
├───────────────────────────────────┼─────────────────────┤
│ AES256 GCM encryption speed       │ 2705.21 MB/s (74%)  │
└───────────────────────────────────┴─────────────────────┘

 

[dcsapak]

the zstd benchmarks are purely cpu, and they don't look so bad (note the percentages in the brackets is relative to our reference machine where most of it was developed a ryzen 2700x AFAIR)

 

[Kodey]

the zstd benchmarks are purely cpu, and they don't look so bad (note the percentages in the brackets is relative to our reference machine where most of it was developed a ryzen 2700x AFAIR)

Thanks. Those zstd figures look pretty low compared with most of the others I've seen here.
The encryption speed looks much better than many of the others I've seen here but It's quite low compared with the reference machine.
What could be holding them up?

 

[Neobin]

Older ZStd benchmarks were (way) higher than the newer ones: [1].

Would be interesting to know why.

[1] https://forum.proxmox.com/threads/how-fast-is-your-backup-datastore-benchmark-tool.72750

 

[dcsapak]

ah that, yeah we changed the testdata at one point: https://git.proxmox.com/?p=proxmox-...ff;h=baae780c997dfb0ce89eae79606619bebd1f4b1f

reason is that we used pure random data before, which is not really compressable and zstd is able to detect that fast and skips compression.
now we use testdata which is compressible, so the results are more realistic

in general i'd try to compare with the percentage since we'll update that even when the underlying testdata/algorithm would change