hint: ZFS w/ lz4 vs. logrotate gzip

[morph027]

Me being a curious guy which loves ZFS since years, i was installing ZFS on my Desktop to (slightly guided by this). I've stumbled upon an interesting statement:

As /var/log is already compressed by ZFS, logrotate’s compression is going to burn CPU and disk I/O for (in most cases) very little gain. Also, if you are making snapshots of /var/log, logrotate’s compression will actually waste space, as the uncompressed data will live on in the snapshot. You can edit the files in /etc/logrotate.d by hand to comment out compress, or use this loop (copy-and-paste highly recommended):

...

Ok, sounds legit. To apply to Proxmox and LXC Containers on ZFS, i've used this handy snippet ($TANK is the ZFS dataset which holds your containers):

find $TANK -regextype sed -regex ".*/etc/logrotate.d/.*" \
-exec grep -Eq "(^|[^#y])compress" {} \; \
-exec sed -i -r "s/(^|[^#y])(compress)/\1#\2/" {} \;

Good to know that the second -exec will only be executed if the first succeeds.

 

[LnxBil]

Yes, I wrote myself an apt hook for that and packaged it:

root@machine ~ > cat /etc/apt/apt.conf.d/logrotate_compress_apt_hook
DPkg::Post-Invoke {"/usr/share/kvm-vm/logrotate-nocompress-hook.sh";};

root@machine ~ > cat /usr/share/kvm-vm/logrotate-nocompress-hook.sh
#!/bin/bash

# disable logfile compression for better deduplicatability
for i in /etc/logrotate.d/*
do
    sed -i -E 's/(\s+)(delay|)compress/\1#\2compress/' $i
done

If a package gets updated, the files are now automatically repatched to work. I also added the dateext flag such that the logfiles are now only renamed once and have the same name until they're deleted.

 

[LnxBil]

Oh, I also added (and packaged) another logrotate goddie:

preremove
    test -f "$1" && shred --iteration=1 --zero "$1"
endscript

deleting the file by zeroing it out. This saves ZFS storage space and yields better backup times.

 

[morph027]

Nifty, thanks. Going to add this to my templates!

 

[morph027]

https://gitlab.com/packaging/logrotate-no-compress-apt-hook