Replication with compression

[Benoît Fontaine]

Hi,
Is there a way through proxmox (pve 5) to replicate with compression ?

Something like :

zfs send -Rpvi tank/vm-100-disk-0@__replicate_100-0_1538395933_ tank/vm-100-disk-0@__replicate_100-0_1538474876_ | bzip2 -c | ssh node2 "bzcat | zfs recv -F tank/vm-100-disk-0"

 

[Knuuut]

man zfs:

       -c, --compressed
           Generate a more compact stream by using compressed WRITE records for blocks which are compressed on disk and in memory (see the compression property for
           details).  If the lz4_compress feature is active on the sending system, then the receiving system must have that feature enabled as well.  If the
           large_blocks feature is enabled on the sending system but the -L option is not supplied in conjunction with -c, then the data will be decompressed before
           sending so it can be split into smaller block sizes.

also dedup might work:

       -D, --dedup
           Generate a deduplicated stream.  Blocks which would have been sent multiple times in the send stream will only be sent once.  The receiving system must also
           support this feature to receive a deduplicated stream.  This flag can be used regardless of the dataset's dedup property, but performance will be much better
           if the filesystem uses a dedup-capable checksum (for example, sha256).

 

[LnxBil]

In addition to @Knuuut s answer:

- bzip2 is single thread bound, all ZFS operations are parallelized including the compression if done on the ZFS side
- SSH also includes a compression option, please refer to the manpage
- SSH is also encryption bound to one cpu core, so the best way to load off data is to have parallel compression and parallel encryption

 

[Benoît Fontaine]

Thanks, zfs send -c seems very useful. But is there a way to configure this option through proxmox web interface ? Or in /etc/pve/replication.cfg ?

 

[LnxBil]

Thanks, zfs send -c seems very useful. But is there a way to configure this option through proxmox web interface ? Or in /etc/pve/replication.cfg ?

I don't think so.

However, for the SSH compression, this can be used in /root/.ssh/config for all SSH connections.

 

[Benoît Fontaine]

I found a solution : update the file /usr/share/perl5/PVE/Storage/ZFSPoolPlugin.pm line 672 (pve 5.2-9):
replace

    my $cmd = ['zfs', 'send', '-Rpv'];

by

    my $cmd = ['zfs', 'send', '-RpvcD'];

It is not necessary to restart any service.

It's not a very clean solution (and it can possibly be deleted if you update your proxmox), but it works.
I think that SSH compression can be useful in addition.

 

[LnxBil]

I think that SSH compression can be useful in addition.

Using compression in addition to another compression will in general not lead to smaller files.

To get your patch into mainline PVE, just open a feature request and provide them with a patch.

 

[Benoît Fontaine]

I just added a patch on an existing feature request :
https://bugzilla.proxmox.com/show_bug.cgi?id=1824