[SOLVED] Should you use compressed ZFS Pools?

Cookiefamily

Well-Known Member
Jan 29, 2020
136
37
48
Germany
Hello,

I was wondering if using the compression feature on ZFS Pools is worth it. The press release of PBS states the following:
Proxmox Backup Server supports incremental backups, data deduplication, compression, and authenticated encryption.
And in Proxmox you can enable ZSTD compression. Usually compressing already compressed data doesn't really work that well and just creates overhead, has someone done performance testing with PBS+ZFS yet?
 
well i use zfs on pbs and I cannot see any disadvantage using zfs compression, the compression ratio is around 1.03 and it's certainly not slower than compression=off
 
the disadvantage is that it's using more cpu & more power (=more cost for energy and more co2)
i think it's difficult to tell whats worse here - wasting a really tiny bit of storage or cpu cycles....
 
the disadvantage is that it's using more cpu & more power (=more cost for energy and more co2)
i think it's difficult to tell whats worse here - wasting a really tiny bit of storage or cpu cycles....
well, we can agree to disagree then. I still think there are no disadvantages, and our DC runs on wind energy so no co2 thoughts there, and for the cost, well, better run no backups then ;)

a quick test with zstd compressed copy with or without compression shows really tiny cpu usage compared to the overall usage:
zfs compression on cputime=12.55s
root@backupserver1:/backups/test# time cp pwned.zstd test.zstd
real 2m9.319s
user 0m0.053s
sys 0m12.497s

zfs compression off cputime=12,43s
root@backupserver1:/backups/test# time cp pwned.zstd test.zstd
real 2m15.074s
user 0m0.069s
sys 0m12.361s
 
well, we can agree to disagree then. I still think there are no disadvantages, and our DC runs on wind energy so no co2 thoughts there, and for the cost, well, better run no backups then ;)

a quick test with zstd compressed copy with or without compression shows really tiny cpu usage compared to the overall usage:
zfs compression on cputime=12.55s
root@backupserver1:/backups/test# time cp pwned.zstd test.zstd
real 2m9.319s
user 0m0.053s
sys 0m12.497s

zfs compression off cputime=12,43s
root@backupserver1:/backups/test# time cp pwned.zstd test.zstd
real 2m15.074s
user 0m0.069s
sys 0m12.361s
thanks for sharing. that looks better then i thought
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!