Cluster & Live Migration

[ale123]

Hi all.
I'm new to virtualization and proxmox and I have a couple of question:

We would like to set up a two node cluster made with a pair of DELL PowerEdge 2950 with 4x146GB SAS 15k rpm disks.

We have not a SAN, DAS, NAS or a shared storage.
Each server will have it's disks inside.

Is possibile to do live migration of VM1 and VM2 from node1 to node2 without stopping them?

For example: if a customer is downloading a big file in VM1, what will happens if i'll start a live migration from node1 to node2? It will be disconnected?

How does it work? Without a SAN each vm should be transfered between nodes via ethernet, how long does it take and how is possibile that no interruption is made?

What if node1 will fail? Proxmox mantains in sync the two nodes or if one node will fail every VM inside of it will be stopped?

Thank you and sorry for my very bad english but I'm Italian.

 

[dietmar]

Is possibile to do live migration of VM1 and VM2 from node1 to node2 without stopping them?

yes

For example: if a customer is downloading a big file in VM1, what will happens if i'll start a live migration from node1 to node2? It will be disconnected?

no

How does it work? Without a SAN each vm should be transfered between nodes via ethernet,

copy data with rsync

how long does it take

depends on size of virtual disk and RAM.

and how is possibile that no interruption is made?

by save/restore the whole state.

What if node1 will fail? Proxmox mantains in sync the two nodes or if one node will fail every VM inside of it will be stopped?

every VM inside of it will be stopped

 

[ale123]

Thank you for the clear response.
But during rsync copy, what happens to the client? New data are store in new server or in old server?

For example, customers will downlaod a huge file from internet during a live migration. It has a 40GB virtual machine full of file, rsync copy will take a long time.

During this time, new file are saved in the new machine? And older file? If a customers want to access to a older file non yet transfered by rsync, how can do it?

Are there any scheduled rsync copy in proxmox? For example to sync every virtual machine every 6 hours, so the live migration will take a shorter time because it will transfer only the differences instead the whole virtual machine every time.

 

[dietmar]

But during rsync copy, what happens to the client? New data are store in new server or in old server?

we use several rsync passes. During last rsync the VM is suspended.

For example, customers will downlaod a huge file from internet during a live migration. It has a 40GB virtual machine full of file, rsync copy will take a long time.

The only way to copy data is to copy it. We do this as fast as possible. If you want to avoid the suspend time you need to use shared storage (on the readmap).

During this time, new file are saved in the new machine? And older file? If a customers want to access to a older file non yet transfered by rsync, how can do it?

Please read the source code of 'vzmigrate' and 'qmmigrate' if you want to know whats going on exactly - these are simple scripts.

Are there any scheduled rsync copy in proxmox? For example to sync every virtual machine every 6 hours, so the live migration will take a shorter time because it will transfer only the differences instead the whole virtual machine every time.

No, there is no scheduled rsync. Again, we automaticaly do a first rsync while the VM is active. After that, we suspend the VM and do a second rysnc.

If you want faster live migration you need to use shared storage.

 

[ale123]

we use several rsync passes. During last rsync the VM is suspended.

That's not a live migration if a VM is suspended....

EDIT:

is possibile to setup up a two-node cluster with different machine over each node?

Last question: if i'll buy two 8GB servers, and I'll create 8 VM with 1GB ram over each server, in case of migration of 1 server over the second one, is possibile to do ram/cpu 'overselling' ?
For example 16 vm with 1gb ram each, on a server with 8gb ram....

 

[dietmar]

I thing you are confused. This is live migration - all connections remains active and inside the VM you did not notice anything.

You always have a short suspend time - even ony ESX with shared storage. The only question is how long that time is.

- Dietmar

 

[ale123]

Yes, I'm confused. But what does it mean "suspend time" ? During this time is non possibile to use a virtual machin or continue to download a file from internet?

 

[dietmar]

Yes, I'm confused. But what does it mean "suspend time" ? During this time is non possibile to use a virtual machin or continue to download a file from internet?

During that time the VM is 'suspended' - the virtual CPU is stopped. As stated above. you always need to do that to get a consistent copy, even on shared storage clusters (to make a consistent copy of RAM). (some optimizations are possible to minimize that suspend time - using several passes).

Anyways, do you know a better/faster way to do live migration on non-shared storage systems?

 

[ale123]

Let's assume that a customer is downloading a big file from rapidshare or any other service that has a "tocken" for download. After that token, download is expired.

If suspend time will go over the customers token, download shoul be restarted again, right?

Just to know....


is possibile to setup up a two-node cluster with different machine over each node?

Last question: if i'll buy two 8GB servers, and I'll create 8 VM with 1GB ram over each server, in case of migration of 1 server over the second one, is possibile to do ram/cpu 'overselling' ?
For example 16 vm with 1gb ram each, on a server with 8gb ram....

 

[dietmar]

is possibile to setup up a two-node cluster with different machine over each node?

I do not understand that question?

Last question: if i'll buy two 8GB servers, and I'll create 8 VM with 1GB ram over each server, in case of migration of 1 server over the second one, is possibile to do ram/cpu 'overselling' ?
For example 16 vm with 1gb ram each, on a server with 8gb ram....

Yes, as long as the VM does not use more that 8GB. But overcommiting resources is a bad practice IMO and can lead to unexpected errors.

 

[ale123]

I do not understand that question?

two node cluster.
node 1 with VM1, VM2, VM3
node 2 with VM4, VM5, VM6

six different virtual machine and not 3 virtual machine with failover capability only.

Yes, as long as the VM does not use more that 8GB. But overcommiting resources is a bad practice IMO and can lead to unexpected errors.

I know. But if one node will fail I need to migrate virtual machine from that node to another node until first node will goes up again.

Let's assume that all 8 virtual machine over the first node are made with 1gb.

Every cluster node has a total of 8gb ram on it.

When I need to migrate VM from the first node to the second node, the second node should have 16GB. But if i've got only 8GB? Are there any memory resizing function? For example resize every virtual machine that will be migrated to 512M instead 1GB until the first node is restored.

 

[dietmar]

two node cluster.
node 1 with VM1, VM2, VM3
node 2 with VM4, VM5, VM6

six different virtual machine and not 3 virtual machine with failover capability only.

We do not provide automatic failover at all.

When I need to migrate VM from the first node to the second node, the second node should have 16GB. But if i've got only 8GB? Are there any memory resizing function? For example resize every virtual machine that will be migrated to 512M instead 1GB until the first node is restored.

no, because thats a quite dangerous task (A VM immediatley dies if you shrink memory too much)

 

[ale123]

So, in a two node cluster environment, with 8 gb ram on each cluster node and 8 machine (1gb ram) over each cluster node, is not possibile to migrate ALL virtual machine from one node to the other node due to the lack of ram over the destination node.

In this case, should be better to create a 3 node cluster with more ram and migrate some VM over one node and some VM over another node.

That's right?

Are there any limits to the number of cluster member? Can I make a 2 node cluster and even a 50 node cluster?

 

[dietmar]

So, in a two node cluster environment, with 8 gb ram on each cluster node and 8 machine (1gb ram) over each cluster node, is not possibile to migrate ALL virtual machine from one node to the other node due to the lack of ram over the destination node.

It is possible if you do not use more that 8GB. But as you stated, you VMs can use up to 16GB - so it is possible that you get errors.

In this case, should be better to create a 3 node cluster with more ram and migrate some VM over one node and some VM over another node.

That's right?

yes

Are there any limits to the number of cluster member? Can I make a 2 node cluster and even a 50 node cluster?

There is no limit. The largest cluster tested is with 32 nodes - but the web interface gets unusable/clumsy/slow at that large number of nodes.