ZFS Pool failed drive. Unable to replace

[caleycom]

I have a server, HPE ProLiant DL360 Gen 9 running 8 x 1TB SATA SSD

All has been working well but a drive has failed, no problem I thought as I have XFS I'll just pull the failed on and pop in a new one.

I have followed the guide listed here: https://dannyda.com/2020/05/16/how-...al-disk-from-proxmox-pve-for-zfs-pool-easily/ but keep getting the error:

cannot replace /dev/disk/by-id/ata-ADATA_SSD_DP910_1TB-DL3_8F2440000998-part1 with ata-Samsung_SSD_850_EVO_1TB_S3PLNF0JA21032W: no such device in pool

Some screenshots are attached showing the details of the current pool and list of the new disk. Am I missing something very basic here or should I start backing up the VM's and nuke the whole thing?

The command seems to add the ZFS reserved partitions to the disk, but still fails. The ADATA drives are the old ones and Samsung is the new one. I have already tried a wipe and initialize from the WebUI with no luck

 

[Dunuin]

Did you tried it with zpool replace CC-ZFS1 /dev/disk/by-id/ata-Samsung_SSD_850_EVO_1TB_S3PLNF0JA21032W? Or zpool replace CC-ZFS1 9041272446610799115 /dev/disk/by-id/ata-Samsung_SSD_850_EVO_1TB_S3PLNF0JA21032W?

 

[caleycom]

Did you tried it with zpool replace CC-ZFS1 /dev/disk/by-id/ata-Samsung_SSD_850_EVO_1TB_S3PLNF0JA21032W? Or zpool replace CC-ZFS1 9041272446610799115 /dev/disk/by-id/ata-Samsung_SSD_850_EVO_1TB_S3PLNF0JA21032W?

The second version worked.

Thanks so much. Will spin up a PBS server on an adjacent server this afternoon to keep a hot backup for future.

 

[Dunuin]

And I would force a scrub after the resilvering: zpool scrub CC-ZFS1
Your second picture showed that your got read/write errors on two disks. A raidz1 can only handle errors of a single disk. If you got errors on two or more disks your data might get corrupted. Would be more secure to run a raidz2, especially when only using cheap consumer SSD that aren't recommended anyway when using ZFS.

 

[caleycom]

And I would force a scrub after the resilvering: zpool scrub CC-ZFS1
Your second picture showed that your got read/write errors on two disks. A raidz1 can only handle errors of a single disk. If you got errors on two or more disks your data might get corrupted. Would be more secure to run a raidz2, especially when only using cheap consumer SSD that aren't recommended anyway when using ZFS.

Is it possible to change from RAID-Z1 to Z2 without destroying and rebuilding the array ?

 

[caleycom]

Scrub has came back clean, no errors. in fact all error warning were removed after resilvering but I have also spun up a proxmox backup server in another location and am doing daily backups of core services to it to avoid any data loss in the event of a 2 drive failure in the future

 

[Dunuin]

Is it possible to change from RAID-Z1 to Z2 without destroying and rebuilding the array ?

No.

Scrub has came back clean, no errors. in fact all error warning were removed after resilvering but I have also spun up a proxmox backup server in another location and am doing daily backups of core services to it to avoid any data loss in the event of a 2 drive failure in the future

Raid never replaces a backup, so everyone should do that.