Live Storage Migration Speed

sirtech

New Member
Oct 20, 2012
7
0
1
What is the limiting factor(s) when performing storage migration on a running VM?

When performing storage migration (NFS => NFS backends), we see a 750Mbit/s average transfer speed when doing storage migration on a VM that is switched off. On a running VM we see 300Mbit/s average on the same configuration. The best I have ever been able to achieve on a live storage migration is 450Mbit/s but I can't repeat that, and it wasn't sustained anyway. I have tried this on PVE v3.0, 3.1, and now 3.3 with very little variation in the results. There is negligible other storage network activity for the whole time of the migration (~20Mbit/s), so it's not interference from another network device.

Since I have verified that this is not a hardware limitation, I would like to know what it is about a running VM that is the bottleneck so I can tune the speed of live storage migration.
 
Doing online migration add a huge overhead to the operation since the move requires maintaining the changes to the storage while moving it to somewhere else. This means maintaining a separate image on the new storage for changes made while doing the move so new writes will go to this new image as long as the move is performed. When moving the storage has finished the temporary image used for storing changes while moving has to be merge with the moved storage. So the rule of thumb is that the performance is greatly influenced by the I/O in the VM.
 
Yes, that makes sense. However, I can spin up a VM that has zero load (IO or otherwise), and it still only averages around 300Mbit/s for a live storage migration. Offline storage migration is consistently at least double that (if not more - 750Mbit/s is the average I'm seeing at present).
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!