1. Problem is next: working well Proxmox for home with Plex, Transmission and Samba.
Decided to add 2 new HDDs to the system. Plug in and ZFS pool become unavailable, cause 2 "old" disks in pool was /dev/sda and /dev/sdb. But "new" ones after connection start as /dev/sda and /dev/sdb/
Unplug "new added disks, "old" HDDs returned to their original state, ZFS pool available again but no access to data.
2. Current status for pool is well according to Proxmox, but all documents/photos - not accessible. No catalogs in system, just available unused folders. Status below:
--------------------------------------------------------------------
root@pve:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
nnpoolz 569G 2.96T 569G /nnpoolz
--------------------------------------------------------------------
root@pve:~# zpool list -v
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
nnpoolz 3.62T 569G 3.07T - - 0% 15% 1.00x ONLINE -
wwn-0x50014ee20d82c0be 1.81T 230G 1.59T - - 0% 12.4% - ONLINE //
in the beginning it was just /dev/sda and /dev/sdb,
wwn-0x5000c50078dff321 1.81T 339G 1.48T - - 1% 18.3% - ONLINE //
after all attempts to restore got by dev_id , but no results
--------------------------------------------------------------------
root@pve:~# zpool history
History for 'nnpoolz':
2020-02-21.14:27:02 zpool create -f -o ashift=12 nnpoolz /dev/sda /dev/sdb
2020-02-25.16:09:31 zpool import -c /etc/zfs/zpool.cache -aN
2020-02-27.11:24:14 zpool import -c /etc/zfs/zpool.cache -aN
2020-02-27.16:28:16 zpool import -c /etc/zfs/zpool.cache -aN
2020-02-29.14:54:05 zpool import -c /etc/zfs/zpool.cache -aN
2020-02-29.16:22:47 zpool import -c /etc/zfs/zpool.cache -aN
2020-03-04.21:22:18 zpool import -c /etc/zfs/zpool.cache -aN
2020-03-08.00:24:04 zpool scrub nnpoolz
2020-03-29.18:56:04 zpool import -c /etc/zfs/zpool.cache -aN
2020-04-12.00:24:04 zpool scrub nnpoolz
2020-04-18.19:01:16 zpool import -c /etc/zfs/zpool.cache -aN
2020-05-07.20:32:41 zpool import -c /etc/zfs/zpool.cache -aN
2020-05-10.00:24:04 zpool scrub nnpoolz
2020-05-11.22:44:34 zpool import -c /etc/zfs/zpool.cache -aN
2020-05-22.19:54:37 zpool import -c /etc/zfs/zpool.cache -aN
2020-06-04.20:29:58 zpool import -c /etc/zfs/zpool.cache -aN
2020-06-08.22:13:13 zpool import -c /etc/zfs/zpool.cache -aN
2020-06-14.00:24:09 zpool scrub nnpoolz
2020-07-05.23:53:10 zpool import -c /etc/zfs/zpool.cache -aN
2020-07-12.00:24:05 zpool scrub nnpoolz
2020-07-13.23:07:38 zpool import -c /etc/zfs/zpool.cache -aN
2020-07-23.20:51:21 zpool import -c /etc/zfs/zpool.cache -aN
2020-07-30.22:22:22 zpool import -c /etc/zfs/zpool.cache -aN
2020-08-07.20:22:43 zpool import -c /etc/zfs/zpool.cache -aN
2020-08-09.00:24:09 zpool scrub nnpoolz
2020-09-01.19:30:54 zpool import -c /etc/zfs/zpool.cache -aN
2020-09-05.21:04:52 zpool import -c /etc/zfs/zpool.cache -aN
2020-09-13.00:24:09 zpool scrub nnpoolz
2
020-09-19.18:10:28 zpool import -c /etc/zfs/zpool.cache -aN // new disks added and then unplugged (starts all problem)
2020-09-20.14:30:16 zpool import -c /etc/zfs/zpool.cache -aN
2020-09-20.17:37:10 zpool import -c /etc/zfs/zpool.cache -aN
2020-09-22.12:08:04 zpool export nnpoolz
2020-09-22.12:12:24 zpool export nnpoolz
2020-09-23.15:42:26 zpool import -c /etc/zfs/zpool.cache -aN
2020-09-24.13:50:28 zpool import -c /etc/zfs/zpool.cache -aN
2020-09-28.07:20:47 zpool import -c /etc/zfs/zpool.cache -aN
2020-09-28.11:25:19 zpool import -c /etc/zfs/zpool.cache -aN
2020-09-28.20:13:59 zpool import -c /etc/zfs/zpool.cache -aN
2020-09-28.20:15:15 zpool export nnpoolz
2020-09-28.20:27:11 zpool import -c /etc/zfs/zpool.cache -aN
2020-09-28.21:39:04 zpool export nnpoolz
2020-09-28.21:40:47 zpool export nnpoolz
2020-09-29.11:53:09 zpool export nnpoolz
2020-10-02.20:24:42 zpool import -c /etc/zfs/zpool.cache -aN
2020-10-03.14:30:12 zpool import -c /etc/zfs/zpool.cache -aN
2020-10-03.14:54:00 zpool import -c /etc/zfs/zpool.cache -aN
----------------------------------------------------------------------------------------
Yes, i restarted pve.
it is not normal by adding new disks on a running server that disks change their devpath
absolutely agree. I switch off server (totally), add disks, then start. Got problem with changed paths, switch off again, unplug disks and start server. And found out this problem.
If you need any additional info, or, remote access even - welcome.
Thanks a lot
Dima