Loose ZFS pool

khanhnguyen

New Member
Dec 4, 2019
7
0
1
43
Hi.
I was not able to ssh to one of our nodes in a cluster. I hard reset the server. After restart my ZFS pool "ssdpool" has a "?" in the web ui. I cannot activate the zpool. 2 SSD was bind to "ssdpool" mirrored. I can see both disks with fdisk -l.

Code:
proxmox-ve: 5.4-2 (running kernel: 4.15.18-18-pve)
pve-manager: 5.4-11 (running version: 5.4-11/6df3d8d0)
pve-kernel-4.15: 5.4-6
pve-kernel-4.15.18-18-pve: 4.15.18-44
pve-kernel-4.15.18-7-pve: 4.15.18-27
pve-kernel-4.15.17-1-pve: 4.15.17-9
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-11
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-53
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-13
libpve-storage-perl: 5.0-44
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-3
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-28
pve-cluster: 5.0-37
pve-container: 2.0-39
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-22
pve-firmware: 2.0-6
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 3.0.1-4
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-54
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.13-pve1~bpo2

I tried this: https://forum.proxmox.com/threads/proxmox-not-mounting-zfs-correctly-at-boot.65724/ not solving the problem. What can I do? Thank for helping!

Greetings
Khanh Nguyen
 
zpool status
What's the output?

Hi Ramalama,

zpool status pool: hddpool state: ONLINE scan: scrub repaired 0B in 7h16m with 0 errors on Sun Mar 14 07:40:56 2021 config: NAME STATE READ WRITE CKSUM hddpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 wwn-0x5000cca253d61422 ONLINE 0 0 0 wwn-0x5000cca253d83dc1 ONLINE 0 0 0 errors: No known data errors pool: rpool state: ONLINE scan: scrub repaired 0B in 0h2m with 0 errors on Sun Mar 14 00:26:28 2021 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 sda2 ONLINE 0 0 0 sdf2 ONLINE 0 0 0 errors: No known data errors pool: ssdpool2 state: ONLINE scan: scrub repaired 0B in 0h11m with 0 errors on Sun Mar 14 00:35:55 2021 config: NAME STATE READ WRITE CKSUM ssdpool2 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 wwn-0x500253884028bd85 ONLINE 0 0 0 wwn-0x500253884028b920 ONLINE 0 0 0 errors: No known data errors

zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT hddpool 10.9T 981G 9.92T - 14% 8% 1.00x ONLINE - rpool 464G 20.8G 443G - 43% 4% 1.00x ONLINE - ssdpool2 952G 497G 455G - 35% 52% 1.00x ONLINE -

Screenshot 2021-03-19 at 15.01.18.png

It is /devsda and /dev/sdf.
 
looks like you've allocated sda2 and sdf2 to rpool with a size of 464G, there wouldn't be any space left on sda1 and sdf1 for ssdpool
 
@bobmc Thanks for pointing. I didn't do anything beside hard restarting the proxmox node because it was not reacting to anything and IPMI was not possible.

fdisk -l Disk /dev/sda: 465.8 GiB, 500107862016 bytes, 976773168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: C7085310-6338-485F-8970-08B5D825021C Device Start End Sectors Size Type /dev/sda1 34 2047 2014 1007K BIOS boot /dev/sda2 2048 976756749 976754702 465.8G Solaris /usr & Apple ZFS /dev/sda9 976756750 976773134 16385 8M Solaris reserved 1 Disk /dev/sdf: 465.8 GiB, 500107862016 bytes, 976773168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: 5CAE3BAD-7C44-454D-8231-04D36AF39F46 Device Start End Sectors Size Type /dev/sdf1 34 2047 2014 1007K BIOS boot /dev/sdf2 2048 976756749 976754702 465.8G Solaris /usr & Apple ZFS /dev/sdf9 976756750 976773134 16385 8M Solaris reserved 1

????
 
Hey, making an pool with sda/sdb is the wrong way.
Those letters can change during boots... Use uuids or names instead.
 
@Ramalama: ACK. I inherited the system. I think the pool was create via WebUI where you have only choose the disks. I don't know if you can use uuids in WebUI to create zpool.
Are my data on sda and sdf lost? How can I move sda2 and sdf2 back without loosing data?