[SOLVED] Can't see ZFS pools beetween 2 nodes

[ViktorE]

Hello, Im new to linux and proxmox also.
Tried to be able for migrations and replications between 2 nodes in cluster
But got some problems..
Any advice?

root@tkp-px1:~# pvesm status
zfs error: cannot open 'px2.zfs1': no such pool
zfs error: cannot open 'px2.zfs1': no such pool
could not activate storage 'px2.zfs1', zfs error: cannot import 'px2.zfs1': no such pool available

Name             Type     Status           Total            Used       Available        %
local             dir     active      1883954048          496640      1883457408    0.03%
local-zfs     zfspool     active      1883457572              96      1883457476    0.00%
px1.zfs1      zfspool     active      5704253440       799316268      4904937172   14.01%
px2.zfs1      zfspool   inactive               0               0               0    0.00%
tkp-pbs           pbs     active     11231004032      5008876928      6222127104   44.60%

root@tkp-px2:~# pvesm status
zfs error: cannot open 'px1.zfs1': no such pool
zfs error: cannot open 'px1.zfs1': no such pool
could not activate storage 'px1.zfs1', zfs error: cannot import 'px1.zfs1': no such pool available


Name             Type     Status           Total            Used       Available        %
local             dir     active       900388992             128       900388864    0.00%
local-zfs     zfspool     active       900389032              96       900388936    0.00%
px1.zfs1      zfspool   inactive               0               0               0    0.00%
px2.zfs1      zfspool     active       902299648         1067788       901231860    0.12%
tkp-pbs           pbs     active     11231004032      5008876928      6222127104   44.60%

Also cant migrate vm



2024-11-13 10:42:42 starting migration of VM 146 to node 'tkp-px2' (172.23.33.11)
2024-11-13 10:42:42 found local disk 'px1.zfs1:vm-146-disk-0' (attached)
2024-11-13 10:42:42 copying local disk images
2024-11-13 10:42:42 full send of px1.zfs1/vm-146-disk-0@__migration__ estimated size is 39.7M
2024-11-13 10:42:42 total estimated size is 39.7M
2024-11-13 10:42:42 TIME SENT SNAPSHOT px1.zfs1/vm-146-disk-0@__migration__
2024-11-13 10:42:43 cannot open 'px1.zfs1': dataset does not exist
2024-11-13 10:42:43 cannot receive new filesystem stream: unable to restore to destination
2024-11-13 10:42:43 cannot open 'px1.zfs1/vm-146-disk-0': dataset does not exist
2024-11-13 10:42:43 command 'zfs recv -F -- px1.zfs1/vm-146-disk-0' failed: exit code 1
2024-11-13 10:42:43 command 'zfs send -Rpv -- px1.zfs1/vm-146-disk-0@__migration__' failed: got signal 13
send/receive failed, cleaning up snapshot(s)..
2024-11-13 10:42:43 ERROR: storage migration for 'px1.zfs1:vm-146-disk-0' to storage 'px1.zfs1' failed - command 'set -o pipefail && pvesm export px1.zfs1:vm-146-disk-0 zfs - -with-snapshots 0 -snapshot __migration__ | /usr/bin/ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=tkp-px2' -o 'UserKnownHostsFile=/etc/pve/nodes/tkp-px2/ssh_known_hosts' -o 'GlobalKnownHostsFile=none' root@172.23.33.11 -- pvesm import px1.zfs1:vm-146-disk-0 zfs - -with-snapshots 0 -snapshot __migration__ -delete-snapshot __migration__ -allow-rename 1' failed: exit code 1
2024-11-13 10:42:43 aborting phase 1 - cleanup resources
2024-11-13 10:42:43 ERROR: migration aborted (duration 00:00:02): storage migration for 'px1.zfs1:vm-146-disk-0' to storage 'px1.zfs1' failed - command 'set -o pipefail && pvesm export px1.zfs1:vm-146-disk-0 zfs - -with-snapshots 0 -snapshot __migration__ | /usr/bin/ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=tkp-px2' -o 'UserKnownHostsFile=/etc/pve/nodes/tkp-px2/ssh_known_hosts' -o 'GlobalKnownHostsFile=none' root@172.23.33.11 -- pvesm import px1.zfs1:vm-146-disk-0 zfs - -with-snapshots 0 -snapshot __migration__ -delete-snapshot __migration__ -allow-rename 1' failed: exit code 1

TASK ERROR: migration aborted

 

[dherzig]

Hi ViktorE,

it seems as if px1.zfs1 and px2.zfs1 are not restricted to the nodes that hold them (this can happen depending on how you created these pools). You can check in Datacenter->Storage and there click each of these. There you will find 'Nodes', where you can remove Node 'tkp-px1' from 'px2.zfs1' and Node 'tkp-px2' from 'px1.zfs1'. This should remove the 'pvesm status' errors.

For migration, you will need to select the appropriate storage -- say you have a vm on 'tkp-px1' with its disk on 'px1.zfs1'. If you migrate it to 'tkp-px2', you will need to select 'px2.zfs1' as 'Target Storage' (as there's no 'px1.zfs1' on 'tkp-px2').

Hope this helps!

Best,
Daniel

 

[ViktorE]

Well, I understand my mistake.
I deleted px2.zfs1 from node2 and created new zfs there with same zfs name as in node1 (px1.zfs1)
All works..