Migrate to a new cluster

[Lucian Lazar]

Hi all,
We are currently running an outdated PVE cluster (version 6.4) consisting of 5 nodes. All VMs and containers are using an external NFS share for both disk storage and backups, mounted at /mnt/pve/NFS.
We’ve recently acquired 2 new nodes, and instead of adding them to the existing cluster, we plan to build a fresh cluster running PVE 8.3.
Our proposed migration strategy is as follows:

1. Remove one node from the old cluster, leaving the minimum 3 nodes required to form the new PVE 8.3 cluster.
2. Mount the same NFS storage on the new cluster using the same path and name (/mnt/pve/NFS), in parallel with the old cluster which will continue to access it during the migration.
3. Shut down one VM or container at a time on the old cluster.
4. Copy the corresponding VM/CT configuration file to the new cluster, keeping the same VMID and storage path.
5. Start the VM/CT on the new cluster.
6. Ensure that the same VM/CT will not be started again on the old cluster to avoid conflicts.

Would this approach be viable? In particular, point #2 raises some concerns — is it safe to mount the same NFS storage on two separate clusters simultaneously, assuming that no VM/CT is active on both at the same time?
Backup and restore is not an option, unfortunately the disks are very big and the downtime would be too much.

Thank you in advance for your input!

 

[gurubert]

is it safe to mount the same NFS storage on two separate clusters simultaneously, assuming that no VM/CT is active on both at the same time?

AFAIK this is safe. Best would be to remove the VM/CT config file from /etc/pve of the old cluster.

You may encounter some issues with the virtual hardware version on the new cluster. VMs (especially Windows) may be picky.

 

[Lucian Lazar]

Thank you, so it is safe even if the same mount point is mounted and active at the same time on both clusters?. Good. Removing the VM/CT config file it is sufficient from /etc/pve/xx/100.conf or must be removed from some other paths too?
Thank you again.

 

[gurubert]

Removing the VM/CT config file it is sufficient from /etc/pve/xx/100.conf

Yes, this is sufficient.

 

[Lucian Lazar]

Yes, this is sufficient.

Thanks, much appreciated

 

[Lucian Lazar]

One last question, isn't there some kind of lock or issues with pmxcfs that proxmox uses when scanning storage? Just want to be extra cautious before attempting to mount the same storage id and path to 2 clusters.

 

[VictorSTS]

I've done this a few times already without issues regarding the storage. You must be absolutely careful that the VM isn't started in both clusters at the same time, though. Make sure no "start on boot" or HA configs are in place, so the VM doesn't start automatically for some reason. Also that no backup will be run for that VM (backup starts QEMU to read the disks and you don't want any kind of access from both clusters at once). If needed, even detach the drives from the VM.

AFAIR, PVE 6.4 already had the option to set QEMU version in VM "system" configuration, so virtual hardware doesn't change when using the VM in different / updated QEMU version.

 

[Lucian Lazar]

I've done this a few times already without issues regarding the storage. You must be absolutely careful that the VM isn't started in both clusters at the same time, though. Make sure no "start on boot" or HA configs are in place, so the VM doesn't start automatically for some reason. Also that no backup will be run for that VM (backup starts QEMU to read the disks and you don't want any kind of access from both clusters at once). If needed, even detach the drives from the VM.

AFAIR, PVE 6.4 already had the option to set QEMU version in VM "system" configuration, so virtual hardware doesn't change when using the VM in different / updated QEMU version.

Thank you, will try with a test VM just to be sure. Nice thing about backups, it is one thing that i could probably have missed!