I'm have HDDs 4kn with RaidZ2 and HBA Mode , after more TBs fill , HDDs fault , after reboot back to normal, how fix this?

[iHostART]

Hello ProxMox users ,I know it's not the most professional way to express myself to the community, but I have a problem that I can't understand and I'm at my wits' end
So , I'm have a 24x 14 TB HDD SAS 4kn
I'm put this HDDs in ZFS RaidZ2 , at a certain time, for example after space usage, at 10 TB, or 32 TB, or 40 TB, one or more HDDs begin to appear as "Fault" in zfs status and raid disappears , if I reboot to node, sometimes all HDDs return to normal and raid starts to appear and work all VPS, but sometimes disappears completely and can not recover data If I test the HDDs individually, they have no problem, how can I solve this because I despair
What it's the problem?Bad HDDs??Poor raid configuration??Any help it's very happy

Thanks for your time!

Regards,
Calin

 

[leesteken]

What make and model are your drives? If they use SMR then they might slow down too much after a while and be marked problematic by ZFS. zpool status (whose output when errors occur would be very useful) would show write errors in that case. If it shows different error counts (or CMR drives), then it might be something else entirely.

 

[iHostART]

Hello @leesteken that it's the model of HDDs https://imgur.com/a/GKUrL5k

I'm read this , https://github.com/openzfs/zfs/issues/14734 it's exactly same problem what I'm have my

A user say If i'm turned off dedup start be stop errors , affected this thing the cache?Or broken data? Any experience

Regards

 

[RolandK]

do you have dedup being turned on ?

 

[iHostART]

Hello @Roland Gsell don't know exactly how view if it's turned on or no


Regards,
Calin

 

[iHostART]

Hello again @RolandK it's off

https://imgur.com/a/HILNM4I

Regards

 

[UdoB]

24x 14 TB HDD SAS 4kn
I'm put this HDDs in ZFS RaidZ2

Did you tell us the number of drives per vdev? Is it 24 for a single vdev? Can you show us the output of zpool status (in CODE-Tags please)?

The recommended maximum number of drives per vdev varies around 10 +/- "some", depending on who you ask.

Personally I would not go over 12...

Good luck...