ZFS RAIDZ2 Degraded

[Nefariousparity]

Hello, we recently purchased a new 45DRIVES XL60 server with 10X Seagate Exos Enterprise 16TB Disks. We configured in a raidz2. Recently I have noticed performance being kind of dismal, and checked it in ProxMox GUI? For a system that is about 30 days old wow! Is this something I can just reset a counter? Did I get unlucky and these drives are bad? What is the procedure here?

 

[ness1602]

Replace the drives ASAP. There is a zpool replace command.

 

[Nefariousparity]

Damn this server and drives are like two weeks old. Unbelievable. Migrating data off it now.

 

[ness1602]

Never mind, this usually happens. Drives die in the first 30 days.

 

[Nefariousparity]

This is a server from 45drives.com The XL60. Darn! I have ordered replacement drives, I have not really put much on it yet. We will see what they say tomorrow.

 

[LnxBil]

Maybe the cables? I've never seen a system with so many drive errors. I suspenct there is something else very wrong here.

 

[Nefariousparity]

Thank you, this is a top load box, the company we bought it from specializes in these rigs. Its brand new about 30 days old. The drives come in top loaded. https://www.45drives.com/products/storinator-xl60-configurations.php We have a little bit beefier than the turbo with 256GB of Ram. We only bought it with 10X 16TB's with plans to add more drives. Maybe the drives are not seated correctly? I have reached out to the company, and as of tonight evacuated all the data off.

 

[LnxBil]

Having everything in only one vdev is also not a good idea performance-wise. You'll only have the write-IOPS on one drive.

 

[Dunuin]

I would also think thats a problem with the backplane, HBA. too much vibrations, power or cableing. When 9 of 10 disks are showing write errors usually not 9 new disks are bad.
You could reset the errors with zpool clear bigrust-m.

 

[Nefariousparity]

I hope that is not the case, this is sitting in a rack in a data center on rails etc.

 

[Nefariousparity]

Okie cool? I ran that command and it says this.

 

[Nefariousparity]

Having everything in only one vdev is also not a good idea performance-wise. You'll only have the write-IOPS on one drive.

We understood this. The plan here was to order in packs of 10 16TB drives starting this week as new additional VDEVS. We have other flash storage for more performance, and about 600 bare metal servers sir. Thank you though as I am old school ZFS is new, to us from hardware raid and this is our first bout.

 

[Nefariousparity]

Just to update what happened, we ended up moving the to another bay to test if the backplane or HBA controller. Proxmox handled export and re import fine. WE are going to monitor over the next few days.

 

[LnxBil]

Just to update what happened, we ended up moving the to another bay to test if the backplane or HBA controller. Proxmox handled export and re import fine. WE are going to monitor over the next few days.

Try to put some load on it with e.g. send/receive some snapshots around, so that you will write a lot.

 

[Nefariousparity]

After running some other hardcore backups last night.