Live Migration PVE 6.0

[Jarvar]

Hello Proxmox Community,
I am interested in live migration. Recently I read posted accorss the fron the XCP-ng's website the phrase "Live migrate your VMs all around the world, without interruption"
However I did not find any detailed sources which stipulate that live migration was actually possible to another place around the world. Possible you could have XCP-ng installed as a cluster on the same network around the world, but surely not from one location to another location in the world.

Is this possible? or is this possible with Proxmox? I'm really happy with Proxmox and would like to see if this is possible,
So far what I have done is backup my VM to an NFS storage. Then I would need to transfer from the NFS storage to an offsite location manally (could probably automate it) and then unpack and qmrestore.

Is there another method? could you actually have the live migration occur to another node on a different network?
That would be awesome.
Thank you so much.

 

[LnxBil]

Is this possible?

If you have a stretched cluster with decent enough connection (low-latency, high bandwidth) it is possible out-of-the-box (as with any other hypervisor clustering). The tricky part is the low-latency over a WAN connection over long range. You need to have a lot of money to make that happen and you're limited by the speed of light in fiber optic cables. Using it in different buildings is no problem and a common setup.

could you actually have the live migration occur to another node on a different network?

What is network for you? The easiest setup would be to create a vlan consisting of all nodes and span them over different physical networks. No problem at all if the latency and bandwidth are ok.

 

[Jarvar]

If you have a stretched cluster with decent enough connection (low-latency, high bandwidth) it is possible out-of-the-box (as with any other hypervisor clustering). The tricky part is the low-latency over a WAN connection over long range. You need to have a lot of money to make that happen and you're limited by the speed of light in fiber optic cables. Using it in different buildings is no problem and a common setup.

I'm relatively new to this. Are you saying that with enough money thrown at the issue it can be done? The setup I have of Proxmox right now is a small dental clinic, so I'm not sure they would be willing to spend the funds if it is a lot to do that. It might be cheaper and more efficient to just setup a cluster for backups. However, then all the VMs are stored in the same location. It would be safer to atleast get something offsite. Can this be done with a VPN or something like that or would it still be too slow?
Their connection right now is a business cable connection with something like 100 Mbs down and 10 Mbs upload. It has taken a while to transfer at that speed. For instance I have managed to wittle down the VM for their server down to about 60 GB at this point which can still be transferred within a day. Something like 15 hours. Their internet speed may be changing in the spring though.

I just started looking into things such as having a Cluster over WAN which somebody managed to get working.
With XCP-ng I posted a similar question because it's advertised on their website. Olivierlambert mentioned using their
https://xen-orchestra.com/blog/devblog-3-extending-the-sdn-controller/


What is network for you? The easiest setup would be to create a vlan consisting of all nodes and span them over different physical networks. No problem at all if the latency and bandwidth are ok.

Well right now for example, I have Proxmox setup as node1 in an office and I make backups to a NAS unit which is on the same VLAN. Ideally, I would like to live migrate those VMs onto another Proxmox server which I have at on offsite location. Or eventually, they have another office, and if both Proxmox setups could talk to each other, then they could communicate.

 

[Ricardo Conejo]

I am an old user of Proxmox, (but not an expert at all), beginning with version 1.9 and then 2.4. and 3.x. Live migration worked fine. The system copies the disk to a new storage while keeping the CT active, then it moves the CT itself
One day I updated to version 6.x and there is no longer live migration available, the only mode available is "Restart mode" as far as I know. There might be some reason for that, but previous Proxmox version live migration were fine.

 

[Jarvar]

I am an old user of Proxmox, (but not an expert at all), beginning with version 1.9 and then 2.4. and 3.x. Live migration worked fine. The system copies the disk to a new storage while keeping the CT active, then it moves the CT itself
One day I updated to version 6.x and there is no longer live migration available, the only mode available is "Restart mode" as far as I know. There might be some reason for that, but previous Proxmox version live migration were fine.

Was this cross pool live migration or just within a cluster?

i just saw a tutorial or FAQ about setting up several Proxmox VMs within an initial bare bones setup in order to test out a cluster setup and live migration and so forth.

I think I have it open in one of my tabs on a desktop browser. If I can find it, I’ll post it here. It might be helpful....

 

[Ricardo Conejo]

Was this cross pool live migration or just within a cluster?

..within a cluster migration. Maybe this feature is available somewhere in version 6.x but I have not found how to configure it. Anyway it is not trivial, as it was in previous versions of Proxmox.

 

[Jarvar]

Hello Ricardo,
Here is the link that speaks about testing out the cluster in a nested virtualized environment,
https://pve.proxmox.com/wiki/Nested_Virtualization
https://pve.proxmox.com/wiki/Nested_Virtualization

It is towards the later part of the page, but it might help in testing what happened with version 6.x before testing out on your live clutster.

I have yet to play around with clusters in Proxmox, but I made several using XCP-ng, which seems really easy to add different nodes to their Xen Orchestra web management tool. I haven't actually backed up or migrated anything yet though.

 

[LnxBil]

Their connection right now is a business cable connection with something like 100 Mbs down and 10 Mbs upload.

That's the bandwidth part and it is very slow. You did not write about latency, which is the crucial part.

The setup I have of Proxmox right now is a small dental clinic, so I'm not sure they would be willing to spend the funds if it is a lot to do that.

No, I'm talking about a medium sized businesses. They can build and maintain those kind of infrastructure. Normally, the WAN connection between sites is either in corporate hand or rented dark fiber, later is very expensive for e.g. 10 GBE. It can easily cost 10k a month.