Loose ZFS pool

khanhnguyen

New Member
Dec 4, 2019
7
0
1
42
Hi.
I was not able to ssh to one of our nodes in a cluster. I hard reset the server. After restart my ZFS pool "ssdpool" has a "?" in the web ui. I cannot activate the zpool. 2 SSD was bind to "ssdpool" mirrored. I can see both disks with fdisk -l.

Code:
proxmox-ve: 5.4-2 (running kernel: 4.15.18-18-pve)
pve-manager: 5.4-11 (running version: 5.4-11/6df3d8d0)
pve-kernel-4.15: 5.4-6
pve-kernel-4.15.18-18-pve: 4.15.18-44
pve-kernel-4.15.18-7-pve: 4.15.18-27
pve-kernel-4.15.17-1-pve: 4.15.17-9
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-11
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-53
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-13
libpve-storage-perl: 5.0-44
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-3
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-28
pve-cluster: 5.0-37
pve-container: 2.0-39
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-22
pve-firmware: 2.0-6
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 3.0.1-4
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-54
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.13-pve1~bpo2

I tried this: https://forum.proxmox.com/threads/proxmox-not-mounting-zfs-correctly-at-boot.65724/ not solving the problem. What can I do? Thank for helping!

Greetings
Khanh Nguyen
 
zpool status
What's the output?

Hi Ramalama,

zpool status pool: hddpool state: ONLINE scan: scrub repaired 0B in 7h16m with 0 errors on Sun Mar 14 07:40:56 2021 config: NAME STATE READ WRITE CKSUM hddpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 wwn-0x5000cca253d61422 ONLINE 0 0 0 wwn-0x5000cca253d83dc1 ONLINE 0 0 0 errors: No known data errors pool: rpool state: ONLINE scan: scrub repaired 0B in 0h2m with 0 errors on Sun Mar 14 00:26:28 2021 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 sda2 ONLINE 0 0 0 sdf2 ONLINE 0 0 0 errors: No known data errors pool: ssdpool2 state: ONLINE scan: scrub repaired 0B in 0h11m with 0 errors on Sun Mar 14 00:35:55 2021 config: NAME STATE READ WRITE CKSUM ssdpool2 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 wwn-0x500253884028bd85 ONLINE 0 0 0 wwn-0x500253884028b920 ONLINE 0 0 0 errors: No known data errors

zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT hddpool 10.9T 981G 9.92T - 14% 8% 1.00x ONLINE - rpool 464G 20.8G 443G - 43% 4% 1.00x ONLINE - ssdpool2 952G 497G 455G - 35% 52% 1.00x ONLINE -

Screenshot 2021-03-19 at 15.01.18.png

It is /devsda and /dev/sdf.
 
looks like you've allocated sda2 and sdf2 to rpool with a size of 464G, there wouldn't be any space left on sda1 and sdf1 for ssdpool
 
@bobmc Thanks for pointing. I didn't do anything beside hard restarting the proxmox node because it was not reacting to anything and IPMI was not possible.

fdisk -l Disk /dev/sda: 465.8 GiB, 500107862016 bytes, 976773168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: C7085310-6338-485F-8970-08B5D825021C Device Start End Sectors Size Type /dev/sda1 34 2047 2014 1007K BIOS boot /dev/sda2 2048 976756749 976754702 465.8G Solaris /usr & Apple ZFS /dev/sda9 976756750 976773134 16385 8M Solaris reserved 1 Disk /dev/sdf: 465.8 GiB, 500107862016 bytes, 976773168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: 5CAE3BAD-7C44-454D-8231-04D36AF39F46 Device Start End Sectors Size Type /dev/sdf1 34 2047 2014 1007K BIOS boot /dev/sdf2 2048 976756749 976754702 465.8G Solaris /usr & Apple ZFS /dev/sdf9 976756750 976773134 16385 8M Solaris reserved 1

????
 
Hey, making an pool with sda/sdb is the wrong way.
Those letters can change during boots... Use uuids or names instead.
 
@Ramalama: ACK. I inherited the system. I think the pool was create via WebUI where you have only choose the disks. I don't know if you can use uuids in WebUI to create zpool.
Are my data on sda and sdf lost? How can I move sda2 and sdf2 back without loosing data?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!