LZO vs ZSTD

[psionic]

I performed a backup test on a 20GB LXC to compare the newly added ZSTD with LZO ( I used 'Mode: Stop' for both):

LZO	ZSTD
INFO: Total bytes written: 6240829440 (5.9GiB, 100MiB/s) INFO: archive file size: 2.80GB INFO: Finished Backup of VM 8114 (00:01:14)	INFO: Total bytes written: 6240747520 (5.9GiB, 79MiB/s) INFO: archive file size: 2.06GB INFO: Finished Backup of VM 8114 (00:01:30)

I ran each algo 3 times and posted the best of each 3.
I definitely see an archive size improvement of approx. 25% with ZSTD, but LZO still won the speed race By approx. 17%.
Are there any tweaks that can be done to improve ZSTD speed?

 

[t.lamprecht]

Are there any tweaks that can be done to improve ZSTD speed?

Well, zstd uses by default an extra IO thread and one thread for compression.
You can increase the threads used for compression by adding a (for example) zstd: 4 line to /etc/vzdump.conf and see if improves things.
Compression level is something which could be have an impact. We do not expose this directly at the moment, though.

The defaults are intended to ensure that the whole PVE host and it's virtual guests do not get slowed down, while still being quite efficient and also fast.

 

[mir]

Well, zstd uses by default an extra IO thread and one thread for compression.
You can increase the threads used for compression by adding a (for example) zstd: 4 line to /etc/vzdump.conf and see if improves things.
Compression level is something which could be have an impact. We do not expose this directly at the moment, though.

The defaults are intended to ensure that the whole PVE host and it's virtual guests do not get slowed down, while still being quite efficient and also fast.

What about option --threads=0..200? Any benchmarks performed adjusting the number of threds used?

There is also this option which could reduce the archive size further and increase decompression time:
--[no-]sparse

 

[t.lamprecht]

What about option --threads=0..200?

Just FYI, the zstd option translate's mostly 1:1 to threads, just for the "0" value we go for half of the available CPU cores as #.

 

[LnxBil]

Any benchmarks performed adjusting the number of threds used?

I did a lot of them back in the days I wrote the bugreport to include zstd, one rough comparison is in the bugreport itself.

a more detailed one (with german headers):

 

[mksa12]

How to use one of the commands from this table to the configuration file /etc/vzdump.conf
in terms of how to configure the vzdump.conf file to use for example zstd -q -c -T16 -1 ?
or how to do it step by step I am green in this

 

[mksa12]

bump

 

[mksa12]

Anyone have any opinions on how to set this up with the panel ?

 

[fabian]

we don't expose anything besides the thread count..