[SOLVED] Attaching Disk to ZFS Pool

[R0ryS]

Greetings All,


I need some help on a Proxmox VE 5.4 system.


The system has 4 x 2Tb SATA hard drives. These have been configured into 2 mirrored pools.
In the boot pool one of the disks failed. This was replaced and resilvered successfully using “/dev/sda”.

Research on this subject revealed that the pool should be created using “by-id” as this is more reliable.

Found an article https://forum.proxmox.com/threads/how-to-import-zfs-root-pool-by-id.24295/ in which @jim.bond.9862 commented about changing the “dev/sdx” mounting to the more reliable “by-id” method.

After doing the “zpool detach rpool sda2” step, I tried to reattach the disk using:

“zpool attach -f rpool ata-ST2000NM001A-2J2101_WS10XE8A-part2” which fails with error
“missing <new_device> specification usage: attach [-f] [-o property=value] <pool> <device> <new-device>” tried with and without the “-f” switch.

AND

“zpool attach rpool sda2 ata-ST2000NM001A-2J2101_WS10XE8A-part2” which fails with error message
“cannot open 'ata-ST2000NM001A-2J2101_WS10XE8A-part2': no such device in /dev must be a full path or shorthand device name”.

Below is some information about the system from “ls -la /dev/disk/by-id: and “zpool status -v”

ls -la /dev/disk/by-id


AND

“zpool status -v



Some direction on this matter will be greatly appreciated.

 

[Dunuin]

I always use /dev/disk/by-id/ata-ST2000NM001A-2J2101_WS10XE8A-part2 and not just "ata-ST2000NM001A-2J2101_WS10XE8A-part2" to define which disk/partition to use when expanding/creating a pool.
Also make sure to not just to add another disk. When replacing a disk from a pool you are booting from there are more steps like cloning the partition table and writing the bootloader. See here paragraph "Changing a failed bootable device": https://pve.proxmox.com/wiki/ZFS_on_Linux#_zfs_administration

 

[R0ryS]

I always use /dev/disk/by-id/ata-ST2000NM001A-2J2101_WS10XE8A-part2 and not just "ata-ST2000NM001A-2J2101_WS10XE8A-part2" to define which disk/partition to use when expanding/creating a pool.
Also make sure to not just to add another disk. When replacing a disk from a pool you are booting from there are more steps like cloning the partition table and writing the bootloader. See here paragraph "Changing a failed bootable device": https://pve.proxmox.com/wiki/ZFS_on_Linux#_zfs_administration

@Dunuin Thank you for the feedback.

Just to clarify. After adding the new disk, the following was done:
sgdisk /dev/sdb -R /dev/sda
sgdisk -G /dev/sda
zpool replace rpool ata-ST2000NM001A-2J2101_WS10XN97 /dev/sda2

“zpool status” displayed “mirror-0”, which is bootable and “mirror-1”
After running “zpool detach rpool sda2” the “mirror-0” is no longer displayed in “zpool status” as per screenshot above.

Does the “mirror-0” pool need to be re-created before adding the disk?

 

[Neobin]

“zpool attach -f rpool ata-ST2000NM001A-2J2101_WS10XE8A-part2” which fails with error
“missing <new_device> specification usage: attach [-f] [-o property=value] <pool> <device> <new-device>” tried with and without the “-f” switch.

As the error already mentions, you also need to specify the existing device/disk to which the new device/disk should be attached to.

zpool attach [-f] <YourPool> <ExistingDisk> <NewDisk>

See example 4-9:
https://docs.oracle.com/en/operatin...ching-and-detaching-devices-storage-pool.html

 

[R0ryS]

Thank you to both @Dunuin & @Neobin for your help and direction.

Just a note for others on what happened.

Read a post about hard disks in the ZPOOL should be mounted as “/dev/disks/by-id/ata-ST2000NM0033-9ZM175_XXXXXXX” and not “/dev/sdaX”. The article referenced above gave directions to do a “detach” and “attach”. The “detach” removed the disk and the “mirror-0” pool.
Due to my lack of experience with ZFS and that the system was booting off “GRUB”, the worry was that attaching the disk would break the boot disk and give me hours of unnecessary work. That was not the case. Below are the steps I took.

List the disks
ls -alh /dev/disk/by-id/

Check the ZFS pool.
zpool status

Detach the disk
zpool detach “poolname” sdaX

Attach the disk
zpool attach -f <pool> </dev/disks/by-id/ata-ST2000NM0033-9ZM175_Exsisting_Disk”> </dev/disks/by-id/ata-ST2000NM0033-9ZM175_New_Disk”>

Wait for resilvering to complete.

 

[sush]

Wait for resilvering to complete.

Any idea how long it took for resilvering to complete?
Ideally it should be 0, but I'm guessing it will still do a full resilvering thinking it is a new disk

 

[alexskysilk]

changing the device name type for a pool is relatively simple-
sudo zpool export [pool name] sudo zpool import -d /dev/disk/by-id [pool name]

Since your device is the boot device you are doing what you need to, which is to detach a drive and attach it back. just omit the part# at the end, zfs will figure out what you want to do.

 

[R0ryS]

Any idea how long it took for resilvering to complete?
Ideally it should be 0, but I'm guessing it will still do a full resilvering thinking it is a new disk

Hey Sush,
The resilvering took about 8 hours. Disks are 2 TB.