Moving Large VM's Between Hosts - Too Big to Backup

[JakeBikeIT]

Hi All,

I have two hosts (not three for a cluster)
One of the machines has a data repository of 20TB -
I need to move this machine to host with larger storage and without a problem with the iLO ...doh..

I have successfully done this with a cluster - but because I only had two nodes I was then unable to clear up the mess.

What is the accepted and trackable method of moving the data to the new host please?

 

[BobhWasatch]

So...you have 20 TB of data that isn't backed up. I guess it isn't very important. You could just throw it away.

 

[JakeBikeIT]

So...you have 20 TB of data that isn't backed up. I guess it isn't very important. You could just throw it away.

Very good - but not particularly helpful.

I am an engineer of long experience and standing believe it or not.

Not backing up that data - is not my decision.

I am required to move the VM from a machine with a minor board fault.

 

[billy999]

If you need it 1 to 1 I would do a vzdump to your new host and then restore it. Since youve got no space left you would need to mount NFS/CIFS to you old host to set the dump filepath.

If you just need the data, I would create a fresh VM on the new host with everything set up and then simply rsync/robocopy the data

 

[spirit]

you can use cli with

qm remote-migrate <vmid> [<target-vmid>] <target-endpoint> --target-bridge <string> --target-storage <string> [OPTIONS]

 

[JakeBikeIT]

you can use cli with

qm remote-migrate <vmid> [<target-vmid>] <target-endpoint> --target-bridge <string> --target-storage <string> [OPTIONS]

Do you get indication of progress please?

 

[esi_y]

So...you have 20 TB of data that isn't backed up. I guess it isn't very important. You could just throw it away.

Horrible people on this forum.

Do you get indication of progress please?

You kind of do, but the feature is considered experimental [1] still by the man page (or "technology preview"). You see output in another thread [2] (NB I do not know what was wrong for the OP there).

And what do you care about progress, watch du -s . on the target?

Are these ZFS pools? You can use zfs send / receive to get that across nicely.

[1] https://pve.proxmox.com/pve-docs/qm.1.html
[2] https://forum.proxmox.com/threads/qm-remote-migrate-error.133741/

 

[JakeBikeIT]

Horrible people on this forum.



You kind of do, but the feature is considered experimental [1] still by the man page (or "technology preview"). You see output in another thread [2] (NB I do not know what was wrong for the OP there).

And what do you care about progress, watch du -s . on the target?

Are these ZFS pools? You can use zfs send / receive to get that across nicely.

[1] https://pve.proxmox.com/pve-docs/qm.1.html
[2] https://forum.proxmox.com/threads/qm-remote-migrate-error.133741/

Well ,,,,

Sometimes there are commercial reasons for not backing up Data - its not of commercial importance - its useful.
And its also run on capacious hardware raid with redundancy - the "dying" host has an irreparable firmware issue - as a box it'll never actually die.

It seems annoying to me that this feature works perfectly well when you create a two node cluster - done it in test.

However- after doing so - you cannot break the cluster apart and delete the remnants of the cluster -- that I've also tried.

 

[esi_y]

It seems annoying to me that this feature works perfectly well when you create a two node cluster - done it in test.

However- after doing so - you cannot break the cluster apart and delete the remnants of the cluster -- that I've also tried.

Ok, so migration works in a cluster of course, but - do you happen to have this using ZFS? If so, why not move the volume (snapshot) manually?

NB It is not that difficult to remove a node from cluster, but it's not really something I would bother going through for this issue.

 

[BobhWasatch]

Horrible people on this forum.

If there was a backup the problem @JakeBikeIT has would not be a problem. Simply restore to the new machine. Instead, the data is "not of commercial" importance, so no backups needed. Yet it is important enough to waste an employee's time moving it around.

I don't think I'm the horrible person here.

 

[esi_y]

If there was a backup the problem @JakeBikeIT has would not be a problem. Simply restore to the new machine. Instead, the data is "not of commercial" importance, so no backups needed. Yet it is important enough to waste an employee's time moving it around.

I don't think I'm the horrible person here.

I didn't mean it personally! The comment was quite killing it for me in a good way. It gets the point across, that's for sure. But for me it's not that simple. I can e.g. imagine having 20TB dataset with lots of snapshots which I want to move around. Then maybe I have a backup but the "snapshots" in those are further apart and that's for a different purpose. It's not a disaster to lose the 1 minute snapshots, but it's nice to retain them too. Just an example.

 

[JakeBikeIT]

If there was a backup the problem @JakeBikeIT has would not be a problem. Simply restore to the new machine. Instead, the data is "not of commercial" importance, so no backups needed. Yet it is important enough to waste an employee's time moving it around.

I don't think I'm the horrible person here.

This is where Proxmox is saving my bacon - the earlier hardware with huge drive capacity is relatively cheap even bought with a warranty - running Proxmox on it means we can run the later version of Microsoft's OS with no issues.