compression/thin provisioning awareness for inter cluster online migration with pdm?

[mnih]

Hi,

we're running two clusters (8.3.3, subscription repo) which are connected via 10gbe. Migration between both nodes works quite fast (300-400MB/s, most probably ceph on the receiving end is limiting).

All VM disks are stored on ceph (source and target) and have the discard option set.
However the migration seems to not use the benefits of thin provisioning, i.e. disks that are big in maximum size but have only little space used (e.g. a 500GiB vhd with 20GiB space used) will be transferred with the same speed over the whole size of the drive. This happens even if the disk has been fstrimmed before.

Is there any option available to allow inter-cluster migration to migrate empty parts of virtual disks faster, like for example proxmox backup server does? ("reading 2GiB/s, writing 0B/s")

 

[LnxBil]

Is there any option available to allow inter-cluster migration to migrate empty parts of virtual disks faster, like for example proxmox backup server does? ("reading 2GiB/s, writing 0B/s")

You mean besides using PBS for migration? No, not at the moment. You could do ceph internal replication, yet this is not available via the GUI and IMHO currently not used. The migration is currently a simple copy-the-disk-1-to-1 to have no underlying requirements for specific storage technologies.

 

[mnih]

Yes, I was talking about online migration. PBS is fine for offline migration. I understand that currently it is a 1:1 copy. Would it be able to activate some kind of compression for the endless zeroes to be transferred quicker? Or would the receiving ceph still have to write those and therefore stay the bottleneck?

 

[fabian]

1. offline and online migration use very different mechanisms under the hood
2. https://bugzilla.proxmox.com/show_bug.cgi?id=5766 / https://bugzilla.proxmox.com/show_bug.cgi?id=4377 - I don't think there is one filed yet for evaluating compression for NBD transfers as used during live migration