Enabling an mdev vGPU causes slow live migration.

[bmernz]

I've been experimenting with the vGPUs for a while now, and I was keen to try out the live migration support in the 8.4 series.

I was pleased to find that it worked, however it takes considerably longer to migrate a vm with a vGPU than without. Even causing the process to repeatedly auto-increased downtime to continue migration: 1638400 ms to levels that occasionally cause migrate-set-parameters error: VM 214 qmp command 'migrate-set-parameters' failed - Parameter 'downtime_limit' expects an integer in the range of 0 to (2000 * 1000) ms. Although the migration still completes eventually.

With 1Gb RAM and 1GB vRAM (but idle) the migration takes over 5 minutes:

2025-05-09 15:46:08 xbzrle: send updates to 276551 pages in 6.5 MiB encoded memory, cache-miss 2.73%
2025-05-09 15:46:09 auto-increased downtime to continue migration: 109951162777600 ms
2025-05-09 15:46:09 migrate-set-parameters error: VM 214 qmp command 'migrate-set-parameters' failed - Parameter 'downtime_limit' expects an integer in the range of 0 to (2000 * 1000) ms
2025-05-09 15:46:16 average migration speed: 3.7 MiB/s - downtime 144 ms
2025-05-09 15:46:16 migration completed, transferred 522.2 MiB VM-state (+ 2.7 KiB VFIO-state)
2025-05-09 15:46:16 migration status: completed
waited 10 seconds for mediated device driver finishing clean up
actively clean up mediated device with UUID 00000000-0000-0000-0000-000000000214
2025-05-09 15:46:31 migration finished successfully (duration 00:05:07)
TASK OK

The same VM with the mdev device removed live migrates in 13 seconds:

2025-05-09 15:50:17 average migration speed: 1.0 GiB/s - downtime 123 ms
2025-05-09 15:50:17 migration completed, transferred 488.4 MiB VM-state
2025-05-09 15:50:17 migration status: completed
2025-05-09 15:50:21 migration finished successfully (duration 00:00:13)
TASK OK

Anyone else seeing this behaviour?

I realise this may be a bit light in detail, happy to collect and add info if it piques anyones interest.