I guess that is even faster when using pigz ;-)
Yes, it's a bit faster. But it's not always the speed that you are interested in.
For example in Hetzner you have only 100GB of backup space included with the base contract, so the compression ratio is often more important than the CPU time spent (usually backups run in the night, when the load is low).
bzip2 have a dramatically better compression ratio compared with gzip.
and xz have an even better compression, but it's 10 times slower than gzip.
Here you are the times and size of the compression of an LVM of 10GB with a fresh debian squeeze 64 install on it
Using pigz
Code:
# time (cat /dev/vgs1/vm-100-disk-1 | pigz > vm-100-disk-1.gz)
real 2m24.517s
user 2m56.836s
sys 0m23.460s
-rw-r--r-- 1 root root 187480727 Apr 22 23:45 vm-100-disk-1.gz
Using pbzip2
Code:
# time (cat /dev/vgs1/vm-100-disk-1 | pbzip2 > vm-100-disk-1.bz2)
real 2m35.549s
user 4m4.963s
sys 0m21.649s
-rw-r--r-- 1 root root 161555772 Apr 22 23:36 vm-100-disk-1.bz2
Using xz
Code:
# time (cat /dev/vgs1/vm-100-disk-1 | xz > vm-100-disk-1.xz)
real 20m36.793s
user 19m16.798s
sys 0m27.434s
-rw-r--r-- 1 root root 132791004 Apr 23 00:08 vm-100-disk-1.xz
As you can see the difference between pigz and pbzip2 is not so big (the limit in this case is the disk speed).
I think it's important to offer the user the widest possible range of possibilities. He will choose the one that better fits his needs.
The best would be to check for the availability on the system of all the compressions utilities and if they are installed to let them being choose.
The more comprehensive list of stream compressors in debian that I can recall is: gzip, bzip2, pigz, pbzip2, lbzip2, xz, lzma.
The default, if available, should be pigz or pbzip2 and I'm more favorable with the second both for the higest compression ratio and, more important, for higher recovery success in case of corruption.
Some hints, where to look at? Are there some git repos with pve source?
