One or more devices could not be used because the label is missing or invalid

[tedxuyuan]

zpool status report zfs pool error, but when I check smart of the error disk, no error found.
What should I do? Please help.

root@pve:/# zpool status -v zfspoolz1
  pool: zfspoolz1
 state: DEGRADED
status: One or more devices could not be used because the label is missing or
    invalid.  Sufficient replicas exist for the pool to continue
    functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-4J
  scan: scrub repaired 0B in 3h48m with 0 errors on Sun Dec  9 04:12:04 2018
config:

    NAME                            STATE     READ WRITE CKSUM
    zfspoolz1                       DEGRADED     0     0     0
      raidz1-0                      DEGRADED     0     0     0
        wwn-0x5000c50079297bb6      ONLINE       0     0     0
        wwn-0x5000c5007929913b      ONLINE       0     0     0
        10266552958893128037        UNAVAIL      0     0     0  was /dev/sdc1
    logs
      wwn-0x5000000000000000-part1  ONLINE       0     0     0
    cache
      wwn-0x5000000000000000-part2  ONLINE       0     0     0

errors: No known data errors

root@pve:/# smartctl -H /dev/sdc
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.15.18-1-pve] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

 

[Richard]

Probably the data are corrupt - try (assuming the suspected device is still /dev/sdc):

zpool replace zfspoolz1 sdc

Then resilvering should take place, when running

zpool status

again you will see something like

  pool: zfspoolz1
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
   continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Thu Mar  7 13:25:07 2019
   1.30G scanned out of 23.0G at 111M/s, 0h3m to go
   439M resilvered, 5.66% done
config:

   NAME                       STATE     READ WRITE CKSUM
   dapol                      DEGRADED     0     0     0
     raidz1-0                 DEGRADED     0     0     0
      wwn-0x5000c50079297bb6                  ONLINE       0     0     0
       replacing-1            DEGRADED     0     0     0
         1472193835399965318  UNAVAIL      0     0     0  was /dev/sdc1/old
         sdc                  ONLINE       0     0     0  (resilvering)
       wwn-0x5000c5007929913b                    ONLINE       0     0     0

errors: No known data errors

After a while (when resilvering finished) everything should be ok.

If it does not work you have to replace the disk and do the same again.

 

[jtpavlock]

Probably the data are corrupt - try (assuming the suspected device is still /dev/sdc):

zpool replace zfspoolz1 sdc

Then resilvering should take place, when running

zpool status

again you will see something like

  pool: zfspoolz1
state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
   continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Thu Mar  7 13:25:07 2019
   1.30G scanned out of 23.0G at 111M/s, 0h3m to go
   439M resilvered, 5.66% done
config:

   NAME                       STATE     READ WRITE CKSUM
   dapol                      DEGRADED     0     0     0
     raidz1-0                 DEGRADED     0     0     0
      wwn-0x5000c50079297bb6                  ONLINE       0     0     0
       replacing-1            DEGRADED     0     0     0
         1472193835399965318  UNAVAIL      0     0     0  was /dev/sdc1/old
         sdc                  ONLINE       0     0     0  (resilvering)
       wwn-0x5000c5007929913b                    ONLINE       0     0     0

errors: No known data errors

After a while (when resilvering finished) everything should be ok.

If it does not work you have to replace the disk and do the same again.

Hi, I have the same issue, but the command does not work. Instead it outputs:

/dev/sdh is in use and contains a unknown filesystem.

Status:

# zpool status
  pool: tank
state: DEGRADED
status: One or more devices could not be used because the label is missing or
        invalid.  Sufficient replicas exist for the pool to continue
        functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-4J
  scan: scrub in progress since Mon Nov  4 19:46:57 2019
        2.32T scanned at 15.7G/s, 106G issued at 719M/s, 23.0T total
        0B repaired, 0.45% done, 0 days 09:17:43 to go
config:

        NAME                                   STATE     READ WRITE CKSUM
        tank                                   DEGRADED     0     0     0
          raidz2-0                             DEGRADED     0     0     0
            wwn-0x5000cca252c6bec9             ONLINE       0     0     0
            wwn-0x5000cca252c5481c             ONLINE       0     0     0
            wwn-0x5000cca252c72c21             ONLINE       0     0     0
            wwn-0x5000cca252c69479             ONLINE       0     0     0
            wwn-0x5000cca252c7665b             ONLINE       0     0     0
            5733517112250420833                FAULTED      0     0     0  was /dev/sdh1
            ata-WDC_WD80EFAX-68LHPN0_7SGG7DKC  ONLINE       0     0     0

errors: No known data errors

Any idea where to go from here? This happened while I was replacing a failed drive, so I currently to do not have any other spares.

 

[proxwolfe]

I have the same problem as the OP.

The solution proposed above seems to me to try to repair the faulted disk "in place" (i.e. without replacing; or replacing it with itself). So I am not replacing the disk and try to implement the suggested solution.

But when I issue the command from above (replacing "sdc" with "sdf" in my case)

zpool replace zfspoolz1 sdf

I get

invalid vdev specification
use '-f' to override the following errors:
/dev/sdf1 is part of active pool 'z2pool1'

and when I add -f I get

invalid vdev specification
the following errors must be manually repaired:
/dev/sdf1 is part of active pool 'z2pool1'

So, rightly, ZFS is complaining that the disk I am trying to "repair" is part of a pool. How do I get around this? Or did I misunderstand something and the proposed solution includes replacing the faulted disk?

Thanks.

 

[FrancisS]

Hello,

I had a "similar" problem after a reboot I had two disks reversed and set FAULTY (with label invalid) and I have the same error messages than you with the "replace".

status: One or more devices could not be used because the label is missing or
invalid. Sufficient replicas exist for the pool to continue
functioning in a degraded state.
action: Replace the device using 'zpool replace'.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-4J

sdc ONLINE 0 0 0
7032113212656790552 FAULTED 0 0 0 was /dev/sdd1
15495950591772021340 FAULTED 0 0 0 was /dev/sde1
sdf ONLINE 0 0 0

[0:0:4:0] disk xxxx xxxxxxxxxxx xxxx /dev/sde
[0:0:5:0] disk xxxx xxxxxxxxxxx xxxx /dev/sdd

normally sdd is 0:0:4:0 and sde 0:0:5:0

I corrected the problem with a export/import.
# zpool export xxxxx
# zpool import -d /dev/disk/by-id xxxxx
# zpool clear

It is a bad idea to use "/dev/sdX" (from the GUI) because a disk can change or can be added and /dev/sdX can change.
The path "/dev/disk/by-id" is better to use (probably the proxmox GUI need to be changed ?).

Also I add ed to the file "/etc/default/zfs"

ZPOOL_IMPORT_PATH="/dev/disk/by-id"

Best regards.

Francis

 

[take]

I corrected the problem with a export/import.
# zpool export xxxxx
# zpool import -d /dev/disk/by-id xxxxx
# zpool clear

I'd like to confirm it worked like a charm! The only thing I had to run before exporting the pool was:

systemctl stop proxmox-backup-proxy.service proxmox-backup.service

and of course it is a good idea to create pool using /dev/disk/by-id/ labels