[SOLVED] How to migrate container from one node to another, each with it's local storage

[Chicken76]

So here's the setup in a nutshell:

• two nodes: node1, node2 in a cluster
• cluster is healthy and running the latest version of PVE, all updates applied
• node1 storages: local, local-zfs, pool1
• node2 storages: local, local-zfs, pool2

The question is: how do I move a container that runs on node1 and resides on pool1 to node2 and have it on pool2?

I've thought of moving the storage of the container from pool1 to local-zfs on node1, then migrate it to node2 and then moving its storage to from local-zfs to pool2. Unfortunately the container is bigger than the free space on local-zfs on either node.

 

[fiona]

Hi,
what kind of storages are pool1 and pool2? Assuming the volumes are raw, you can use

pvesm export pool1:<ID>/vm-<ID>-disk-<N>.raw raw+size - | ssh <IP of NODE2> pvesm import pool2:<ID>/vm-<ID>-disk-<N>.raw raw+size -

While the container is shut down you can then move the config from /etc/pve/nodes/node1/lxc/<ID>.conf to /etc/pve/nodes/node2/lxc/<ID>.conf and update the storage ID from pool1 to pool2 for the rootfs and mountpoints. Then it should be possible to start it on the second node. After making sure everything works, you can remove the left over volumes on node1.

 

[Chicken76]

Thank you Fabian_E for the information.

All storages are ZFS pools.

I was thinking of trying to do zfs send | zfs recv with that dataset like this:

root@node1:/# zfs send pool1/subvol-ID-disk-0 | ssh node2 zfs recv pool2/subvol-ID-disk-0

Do you think this would work?

 

[fiona]

Yes, that should work. With format zfs instead of raw+size, pvesm export/import is basically only a wrapper around zfs send/receive, so it doesn't make much of a difference which of the two you use.

 

[Chicken76]

It worked! Thank you Fabian_E!

Marking the thread as solved and posting here the outcome so it may be useful to someone looking to do the same thing.

The only "problem" I encountered was that the zfs send command would not run because the dataset was mounted (even if the container was not running). After I unmounted the dataset it all worked fine.

 

[promok]

And how to do it without zfs?
I have only local storage which is lvm-thin. Proxmox refuse migrating the container even in shutdown/offline state! It says:
ERROR: migration aborted (duration 00:00:00): storage 'ssdpool2' is not available on node 'proxmox1'
Is this possible in GUI if i do not have shared container storage?
Because the container is not visible in RAW LVM-Thin volume. There is no filesystem on that drive.
What is the point in cluster configuration if it cannot move/migrate containers between storages?

 

[fabian]

@promok please keep your questions to a single thread..