A bad SSD is crashing my host: how to replace it with a larger one in a zfs mirror?

[sylar12]

Hello there,

I'm experiencing some randam crash of my proxmox host, possibly related to the use of a bad SSD (Silicon Power A55 512Go) absolutely not recommanded for being used in a zfs pool. But I knew that a bit too late...
Anyway, I'm now waiting for a new disk which will be larger than the initial disk: my zfs pool (which is the rpool of my system) is made of two 512Go disk, and the new one will be 980Go. I know I will be limited to 512Go, but this is not a problem for now.

I've read in the documentation that the correct way to go was:
1/copy the partition table from the 512Go disk to the new 980Go one:
# sgdisk <healthy bootable device> -R <new device> # sgdisk -G <new device>
2/replace the old disk with the new one:
zpool replace -f <pool> <old zfs partition> <new zfs partition>
3/initialize the EFI partition:
# proxmox-boot-tool format <new disk's ESP> # proxmox-boot-tool init <new disk's ESP>

My question is about the first step: since I will copy a partition table from a 512Go disk to a 980Go one, should I expand the last partition in the larger disk to reach the actual total size of it BEFORE adding this partition to the pool? Or is it something I can do easily later, e.g. when changing the first smaller disk to a larger one to have two disks of the same size in the pool?

Thanks for your good advices

 

[udo]

Hi,
you can use the bigger partition. If you set "zpool set autoexpand=on rpool" than your zfs pool will get the new size after the second SSD ist replaced.

Udo

 

[sylar12]

OK, thanks. So no need to anticipate a larger partition on the new disk, it will be automatically expanded once the pool is made of disks with identical size? Nice!

 

[udo]

OK, thanks. So no need to anticipate a larger partition on the new disk, it will be automatically expanded once the pool is made of disks with identical size? Nice!

No,
you must add the new SSD with an partitiontable which use the whole space. Then it will automaticly expanded after the second ssd is replaced.

 

[sylar12]

Sorry, I think I misunderstood your first response.

In short, my first healthy drive (/dev/sda) has the following partition table:
Device Start End Sectors Size Type /dev/sda1 34 2047 2014 1007K BIOS boot /dev/sda2 2048 2099199 2097152 1G EFI System /dev/sda3 2099200 1000215182 998115983 475.9G Solaris /usr & Apple ZFS

So I will create /dev/sdb1 and /dev/sdb2 on my new disk with identicall type and size, but then I must create (or expand) /dev/sdb3 to a larger size to fit the larger size of the disk before actually adding to the pool, is that correct?

 

[udo]

So that's what I was asking, as in step 1 above recopying the partition table would then create a partition of the same size of the smaller disk.

In short, my first healthy drive (/dev/sda) has the following partition table:
Device Start End Sectors Size Type /dev/sda1 34 2047 2014 1007K BIOS boot /dev/sda2 2048 2099199 2097152 1G EFI System /dev/sda3 2099200 1000215182 998115983 475.9G Solaris /usr & Apple ZFS

So I will create /dev/sdb1 and /dev/sdb2 on my new disk with identicall type and size, but then I must create (or expand) /dev/sdb3 to a larger size to fit the larger size of the disk before actually adding to the pool, is that correct?

if you copy the partition table remove simply partition 3 and create an new one (with parted) with start 2099200s and size 100%.
Or create all partitons with parted (34s 2014s; 2048s 2097152s; 2099200s 100%) and set the type afterwards.

Udo

 

[sylar12]

OK I'll do that as soon as I receive my new disk. Thanks!

 

[LnxBil]

from a 512Go disk to a 980Go one

Why do you always write Go instead of GB?

 

[sylar12]

Because I'm French (Giga Octet, instead of Giga Bytes) I have not noticed I was using the french notation, sorry

 

[LnxBil]

Because I'm French (Giga Octet, instead of Giga Bytes) I have not noticed I was using the french notation, sorry

No problem, thanks for the explanation. There is also Octets in english, but this unit is normally used by "the network people".

 

[sylar12]

Well, it's done. I had to delete and recreate the 3rd partition of the new disk to the correct size. I used fdisk, and since I have:
For the 1st disk:

Disk /dev/sda: 476.94 GiB, 512110190592 bytes, 1000215216 sectors
Disk model: Samsung SSD 840
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 0512A3ED-B6FD-4DAA-B423-A9A76FF29A81

Device       Start        End   Sectors   Size Type
/dev/sda1       34       2047      2014  1007K BIOS boot
/dev/sda2     2048    2099199   2097152     1G EFI System
/dev/sda3  2099200 1000215182 998115983 475.9G Solaris /usr & Apple ZFS

And the the 2nd:

Disk /dev/sdb: 894.25 GiB, 960197124096 bytes, 1875385008 sectors
Disk model: KINGSTON SEDC500
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 71508145-51A5-4544-B5B0-78B988270CA6

Device       Start        End    Sectors   Size Type
/dev/sdb1       34       2047       2014  1007K BIOS boot
/dev/sdb2     2048    2099199    2097152     1G EFI System
/dev/sdb3  2099200 1873287822 1871188623 892.3G Linux filesystem

You can see that the partition type of /dev/sdb3 is now "Linux filesystem", while it is "Solaris /usr & Apple ZFS" on the 1st disk. Is it a problem?

 

[Adams-j]

this solution worked for me, I used cfdisk to delete partition 3 and combine all the extra disk space onto partition 3.

 

[Adams-j]

You can see that the partition type of /dev/sdb3 is now "Linux filesystem", while it is "Solaris /usr & Apple ZFS" on the 1st disk. Is it a problem?

same here, seems to be working fine.

 

[sylar12]

Seems to be OK on my side too

 

[LnxBil]

You can see that the partition type of /dev/sdb3 is now "Linux filesystem", while it is "Solaris /usr & Apple ZFS" on the 1st disk. Is it a problem?

No, parition types usually don't matter (except for booting). Linux does normally don't care about them.