Migrating VM's between unequal hardware nodes - local-zfs no such pool available

[Gamerkoe]

I have a server which is running Proxmox without mayor issues. It consists of two Samsung SSD's (SATA Samsung 860 EVO 500gb) which are mirrored into a ZFS pool called 'rpool'. The host Proxmox PVE + multiple VM's are running on it. Now since this single server had some issues, I thought it might be a good idea to have a second node next to it. I watched this Youtube video where a ZFS pool accros two nodes is explained; https://youtu.be/a7OMi3bw0pQ?si=PCSwCBOPU39baTT0&t=740

The plan was to reuse the 500GB SSD's and place one of each into a newly bought node (Optiplex 7090). I bought some new SSD's as boot drives. I will enable replication + HA groups across the nodes to that when one node fails, the second node should pickup the VM's.
I have a spare laptop which I also installed Proxmox PVE on, which will temporarily host the VM's while I reinstall the initial server because I need those 500gb SSD's for the ZFS pool.

I'm in the phase of migrating VM's to the laptop node, but run into the issue that local disks (on local-zfs) cannot be migrated. I mounted a NFS share on my NAS and made a backup to that share. When I want to restore the backup on the laptop, it complains that it cannot find the local-zfs pool. I found multiple topics here describing that on 'Datacenter' you can assign nodes to the ZFS pool and I tried multiple things here. It will not get the local-zfs pool active/online/available for (re)storing data.

Question 1: should I have the ZFS pool assigned to all nodes or not?

Then I tried moving the disks from the local-zfs storage to nfs storage. This works and I'm able to migrate a VM to the laptop node. But on that node, I'm not able to move the disk from the NFS storage back to a local disk.

Question 2: what is now the best way for me to reach the goal of two nodes with the 500gb ZFS pool across those? I don't want to reinstall the complete VM's again, since on those it's quite a lot of manual work to setup the installed applications.

node 'proxmox' is initial server, node 'proxmoxlaptop' is the temporary node and node 'proxmox1' is the new Optiplex. This last will have 'proxmox2' next to it when I reinstall the proxmox node.

Thanks in advance!!

 

[willieman]

I have a server which is running Proxmox without mayor issues. It consists of two Samsung SSD's (SATA Samsung 860 EVO 500gb) which are mirrored into a ZFS pool called 'rpool'. The host Proxmox PVE + multiple VM's are running on it. Now since this single server had some issues, I thought it might be a good idea to have a second node next to it. I watched this Youtube video where a ZFS pool accros two nodes is explained; https://youtu.be/a7OMi3bw0pQ?si=PCSwCBOPU39baTT0&t=740 geometry dash scratch

The plan was to reuse the 500GB SSD's and place one of each into a newly bought node (Optiplex 7090). I bought some new SSD's as boot drives. I will enable replication + HA groups across the nodes to that when one node fails, the second node should pickup the VM's.
I have a spare laptop which I also installed Proxmox PVE on, which will temporarily host the VM's while I reinstall the initial server because I need those 500gb SSD's for the ZFS pool.

I'm in the phase of migrating VM's to the laptop node, but run into the issue that local disks (on local-zfs) cannot be migrated. I mounted a NFS share on my NAS and made a backup to that share. When I want to restore the backup on the laptop, it complains that it cannot find the local-zfs pool. I found multiple topics here describing that on 'Datacenter' you can assign nodes to the ZFS pool and I tried multiple things here. It will not get the local-zfs pool active/online/available for (re)storing data.

Question 1: should I have the ZFS pool assigned to all nodes or not?

Then I tried moving the disks from the local-zfs storage to nfs storage. This works and I'm able to migrate a VM to the laptop node. But on that node, I'm not able to move the disk from the NFS storage back to a local disk.

Question 2: what is now the best way for me to reach the goal of two nodes with the 500gb ZFS pool across those? I don't want to reinstall the complete VM's again, since on those it's quite a lot of manual work to setup the installed applications.

node 'proxmox' is initial server, node 'proxmoxlaptop' is the temporary node and node 'proxmox1' is the new Optiplex. This last will have 'proxmox2' next to it when I reinstall the proxmox node.

Thanks in advance!!

View attachment 87925

Hello, I can explain for you:
1. No, ZFS pools (like local-zfs or rpool) are local storage, tied to the disks physically attached to a node. You cannot directly share a ZFS pool across Proxmox nodes unless you're using a networked ZFS (like over iSCSI or NFS, or using a clustered filesystem like Ceph, which is different from native ZFS replication).

In the Proxmox storage config (/etc/pve/storage.cfg), local ZFS storage should only be defined on the node(s) that physically have access to the pool.

For example:

zfspool: local-zfs
        pool rpool/data
        content images,rootdir
        nodes proxmox

2. You can’t “merge” or span a single ZFS pool across two physical nodes in a supported or safe way via native ZFS. I think you can create a rpool (or better: name it something distinct like vm-pool) on each node individually using one 500GB SSD per Optiplex node. Use Proxmox’s built-in ZFS replication to asynchronously replicate VM disks between these separate pools. Optionally configure HA groups so that when one node fails, VM definitions can start on another node — and if using ZFS replication, you’ll have a recent disk snapshot ready.

 

[Gamerkoe]

I had a spare USB disk laying around which I attached to the temporary laptop node. I created a tempzfs pool on it and that gave me the option to migrate to the NFS share and then back to the tempzfs pool.

Will shuffle the disk configuration in the future somewhere.