[SOLVED] Issue with replacing failing drive in mirror setup - ZFS

[ReverZ]

Hi all,

one of my two boot SDDs that are configured as mirror is probably failing so the rpool is degraded (although the SMART values are OK) :

        NAME                                                      STATE     READ WRITE CKSUM
        rpool                                                     DEGRADED     0     0     0
          mirror-0                                                DEGRADED     0     0     0
            ata-SPCC_Solid_State_Disk_912307951C8F00123456-part3  FAULTED     17     8     0  too many errors
            ata-SPCC_Solid_State_Disk_F45607151C0500134567-part3  ONLINE       0     0     0

I planned to replace the faulty SDD with an identical model (the same brand, type and size). The brand new SDD is attached via an USB adapter to the server.

I used this command to copy the partition table from the existing SDD to the new SDD:

sgdisk /dev/sdd -R /dev/sda

Unfortunately I get the following statements returned:

Caution! Secondary header was placed beyond the disk's limits! Moving the
header, but other problems may occur!

Warning! Secondary partition table overlaps the last partition by
1 blocks!
Try reducing the partition table size by 4 entries.
(Use the 's' item on the experts' menu.)
Aborting write of new partition table.

Are there any suggestions what to do to fix that without harming the mirror setup?

 

[LnxBil]

Are there any suggestions what to do to fix that without harming the mirror setup?

It looks like your replacement disk does not have the same size. Can you check with each disk name (here sda)

fdisk -l /dev/sda

 

[ReverZ]

All three SDDs (2 installed as mirror) and the new one are the same type (512GB)
But the fdisk output seems that there are small difference:

fdisk -l /dev/sda
Disk /dev/sda: 476.94 GiB, 512110190080 bytes, 1000215215 sectors
Disk model: External    
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

fdisk -l /dev/sdd
Disk /dev/sdd: 476.94 GiB, 512110190592 bytes, 1000215216 sectors
Disk model: SPCC Solid State
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes

So the difference is just one sector (new SSD is one sector smaller) but this is enough to make the trouble.

So can I partition the new SSD manually and reduce the size of the second partition by one sector (compared to the existing SSDs) so that the third partition (ZFS mirror) is the same size as this existing SSDs?

 

[ReverZ]

Nobody who faced the same issue during a disk replacement?

 

[leesteken]

Yes, you can partiton the new SSD manually. Please use gdisk for GPT-partitioned drives. The partition alignment is usually 1MB instead of 1 sector. I don't expect any issues with an ESP (second parition) that's 1MB smaller.

 

[ReverZ]

It is so frustrating ...

This is the partition table of the existing SSDs:

Number  Start (sector)    End (sector)  Size       Code  Name
   1              34            2047   1007.0 KiB  EF02
   2            2048         1050623   512.0 MiB   EF00
   3         1050624      1000215182   476.4 GiB   BF01

In order to win the missing block for the partition 3 (mirrored partition) of the new SSD I wanted to achieve the following partition table (partition 2 should be shrinked by 1MiB):

Number  Start (sector)    End (sector)  Size       Code  Name
   1              34            2047   1007.0 KiB  EF02
   2            2048         1048571   511.0 MiB   EF00
   3         1048572      1000213130   476.4 GiB   BF01

Therefore I tried to create the first partition with the following command:

sgdisk -n 1:34:2047 -t 1:ef02 /dev/sda -p
Information: Moved requested sector from 34 to 2048 in
order to align on 2048-sector boundaries.
Could not create partition 1 from 34 to 2047
Could not change partition 1's type code to ef02!
Disk /dev/sda: 1000215215 sectors, 476.9 GiB
Model: External     
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 5EDB62DE-F78D-4A9A-B2C0-DD1187ECE0ED
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 1000215181
Partitions will be aligned on 2048-sector boundaries
Total free space is 1000215148 sectors (476.9 GiB)

So the partition could not be created due to not matching the default alignment.
And I see no option to overwrite the alignment

How the heck did the Proxmox PVE installer create this partition table? And how can I do the same?
I am out of ideas how to fix it.

 

[leesteken]

So the partition could not be created due to not matching the default alignment.
And I see no option to overwrite the alignment

How the heck did the Proxmox PVE installer create this partition table? And how can I do the same?
I am out of ideas how to fix it.

You can change the alignment via the expert menu of gdisk: use x for the expert menu, then l to change the alignment, then m to return to the main menu.

 

[ReverZ]

You can change the alignment via the expert menu of gdisk: use x for the expert menu, then l to change the alignment, then m to return to the main menu.

Thanks a lot!
This actually worked!
For the creation of the first partition I changed the alignment value to "1". Then I switched it back to "2048" and created partition 2 and 3.
Resilvering in Proxmox was then without a problem.
I am still wondering why this was the partitioning of the Proxmox installer.

 

[leesteken]

I am still wondering why this was the partitioning of the Proxmox installer.

The BIOS boot partition can be used by GRUB and is less than a MB and hardly ever written and therefore not aligned. The ESP partition can be used for UEFI boot (with GRUB or systemd-boot). The Proxmox manual does not say why both are always added but I assume it saves on the logic (and testing) in that part of the installer.