throughput vzdump does not increase when using pigz

[gustav]

hi,

vzdump 154 --mode snapshot --dumpdir /vms/data-1/dump/
took ~26 minutes
..
154: 2024-04-12 15:58:38 INFO: transferred 64.00 GiB in 1551 seconds (42.3 MiB/s)
154: 2024-04-12 15:58:38 INFO: archive file size: 52.54GB
154: 2024-04-12 15:58:38 INFO: Finished Backup of VM 154 (00:26:58)

vzdump 154 --mode snapshot --pigz 1 --dumpdir /vms/data-1/dump/
on the same hardware (12 cores) for the same vm also took ~26 minutes (when vzdump was running i could not see any pigz-proc.)
..
154: 2024-04-12 16:59:40 INFO: transferred 64.00 GiB in 1559 seconds (42.0 MiB/s)
154: 2024-04-12 16:59:40 INFO: archive file size: 52.53GB
154: 2024-04-12 16:59:40 INFO: Finished Backup of VM 154 (00:26:16)

in the past i used pigz in other environments and always observed dramatic speedups.

questions: is there anything wrong with my invocation line? how can i use the power of pigz?

thanks for your answer, gustav

 

[VictorSTS]

You are missing the --compress gzip parameter:

 vzdump 154 --mode snapshot --compress gzip --pigz 1 --dumpdir /vms/data-1/dump/

Why not using ZSTD? It's way faster, uses multiple threads automatically and compression ratio is quite good.

 

[gustav]

i feel that pigz is proven safe - think about xz.

 

[gustav]

You are missing the --compress gzip parameter:

 vzdump 154 --mode snapshot --compress gzip --pigz 1 --dumpdir /vms/data-1/dump/

Why not using ZSTD? It's way faster, uses multiple threads automatically and compression ratio is quite good.

thanks for your answer. yes i missed --compress gzip.

closer examination has shown that gzip, pigz, zstd, .. is not the bottleneck as there are only marginal throughput differences observable.

btw: currently i've ended up at:

ssh root@${VmHost} "vzdump ${VmId} --mode snapshot --stdout --mailto ${MailDst} | pigz -p 4 -6" | gpg -er ${GpgUserId} > ${BackupDir}/${VmName}-${VmId}.gz.gpg

gpg is necessary to be consistent with our other backups.

thanks again, gustav

 

[VictorSTS]

Glad you found a solution for your use case!