Container migration does not use (zfs) compression

[masgo]

I think I stumbled uppon a bug related to container migrations.
I have a container running on zfs storage with compression enabled. It's a test server which produces huge amounts of logfiles, which are highly compressible. The disk usage is 250 GB but the referenced size is around 3.000 GB. A compression ratio of 12x.

I tried to migrate the container to a different server, which also uses zfs storage and was very surprised that the job was not done after 250 GB of data was transferred. In fact it went on an on untill I stopped it at 750 GB transferred.

My conclusion is, that the compression get's completley ignored. Looks like the data is read and sent over the network without any compression. Am I right? Can I do something about it?

What about replication? Does it behave the same?

Backup to Proxmox Backup Server seems to utilize the compression just fine. The amount of data transferred over the network, as well as the storage used on the backup server are ~250 GB, as expected.

 

[LnxBil]

In a share-nothing-architecture, all the data is transfered. If you would have replication enabled and do a switch-over, only the changed data since the last replication job will be transfered and it'll be fast. The fastest method is a dedicated or distributedshared storage, which has already the data and only needs to transfer the memory contents of the VM.

 

[masgo]

In a share-nothing-architecture, all the data is transfered. If you would have replication enabled and do a switch-over, only the changed data since the last replication job will be transfered and it'll be fast. The fastest method is a dedicated or distributedshared storage, which has already the data and only needs to transfer the memory contents of the VM.

Obviously all the data needs to be transferred. But zfs uses block-based compression and achieves such a high compression rate. I would expect the migration to also use compression and achieve a similar, if not bigger, compression rate. Obviously no compression is applied.

Viewing a Proxmox cluster as a share-nothing-architecture seems a bit odd to me.

My question regarding replication is, if I enable replication now, will it also try to send 3 TB over LAN or does it use zfs-send or something else which compresses data.

 

[Chicken76]

I'm not very knowledgeable in this but I think compression would be preserved only if zfs send <-> zfs receive would be used, which I don't think Proxmox does.