Errors after resilver

[koalillo]

Hi,

I followed the "changing a failed bootable device" procedure. I did this because smartd started warning me daily about 24 offline uncorrectable sectors (and then went down to 16).

While the resilver, I saw some failures on the *new* drive:

# zpool status -v rpool
pool: rpool
state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Wed Sep  2 19:39:01 2026
1.67T / 3.16T scanned at 38.1M/s, 1.51T / 3.16T issued at 34.3M/s
1.51T resilvered, 47.69% done, 14:00:23 to go
config:

NAME                                                STATE     READ WRITE CKSUM
rpool                                               ONLINE       0     0     0
mirror-0                                          ONLINE       0     0     0
replacing-0                                     ONLINE       6     0     0
ata-ST4000VN000-1H4168_Z306MLNS-part3         ONLINE       7     0     2  (resilvering)
ata-ST4000VN006-3CW104_WW6ASY7Q-part3         ONLINE       0     0     8  (resilvering)
ata-WDC_WD40EFRX-68N32N0_WD-WCC7K5EJKZXR-part3  ONLINE       0     0     0

errors: No known data errors

VN000 is the old drive, for which ZFS showed no errors before the resilver. After the resilver:

  pool: rpool
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P
  scan: resilvered 3.18T in 1 days 03:14:37 with 0 errors on Thu Sep  3 22:53:38 2026
config:

        NAME                                                STATE     READ WRITE CKSUM
        rpool                                               ONLINE       0     0     0
          mirror-0                                          ONLINE       0     0     0
            ata-ST4000VN006-3CW104_WW6ASY7Q-part3           ONLINE       0     0     8
            ata-WDC_WD40EFRX-68N32N0_WD-WCC7K5EJKZXR-part3  ONLINE       0     0     0

errors: No known data errors

(This system is extremely old. I was considering replacing it, but the price explosions lately have prevented me from doing it.)

I wonder what I should do. Is the new drive bad? Bad enough to require another replacement?

Should I start doubting the whole system?

Can I proceed with the drive replacement as-is?

Cheers,

Álex

 

[koalillo]

(Well, I completed the boot loader steps because I feared a reboot or a failure of the remaining drive could have been fatal.)

 

[woma]

Hi Alex,

usually one would start SMART self test "long" for that drive with errors.

smartctl -t long /dev/{name of your disk}

Then after some time (minutes for small NVMe drives but many hours for huge harddisks) check results with
smartctl -l selftest /dev/{name of your disk}

You will see something like this with self test status.

smartctl 7.5 2025-04-30 r5714 [x86_64-linux-7.0.0-30-generic] (local build)
Copyright (C) 2002-25, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF SMART DATA SECTION ===
Self-test Log (NVMe Log 0x06, NSID 0xffffffff)
Self-test status: Extended self-test in progress (5% completed)
No Self-tests Logged

Later on result will show up:

smartctl 7.5 2025-04-30 r5714 [x86_64-linux-7.0.0-30-generic] (local build)
Copyright (C) 2002-25, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF SMART DATA SECTION ===
Self-test Log (NVMe Log 0x06, NSID 0xffffffff)
Self-test status: No self-test in progress
Num  Test_Description  Status                       Power_on_Hours  Failing_LBA  NSID Seg SCT Code
 0   Extended          Completed without error                1003            -     -   -   -    -

"Extended" tests showing status "Completed without error" are a strong indication that your drive is OK.

 

[leesteken]

Checksum errors could be bad drive, bad cable, bad controller or bad system memory. Test each part in the chain and/or replace parts (temporarily) to find the culprit. This is not specific to Proxmox and other Linux/ZFS/hardware troubleshooting guides will probably apply.