zfs write errors

[foolonthehill]

I get write error on sas hdds and sata ssd in zfs
The disks are connected to a hardware raid controller in HBA Mode.

new System:

• proxmox-ve 8.2.0
• Mainboard Supermicro MBD-H12DSI-NT6
• 2 x AMD EPYC MILAN 7313 3GHz, 16 cores
• 256GB RAM
• Adaptec SmartRAID 3152-8i
• ZFS10 with 4 drives

Can i use this RAID controller?
I think, i need ZFS for replication/migration to 2nd identical server.
Is ZFS on top of hardware raid10 a possible solution?
The problem occured after testing the setup for several weeks.
Now it's productive with more erratic disk i/o.
How can i trace the cause of the problem?

thx al lot and have a nice day,

Harry

 

[dakralex]

Welcome to the Proxmox forum, Harry!

Could you provide further details on the erratic disk I/O? What does exactly happen?

In general, ZFS on a hardware raid controller is not recommended nor supported as described at [0] and other spots in the Proxmox VE admin guide. Does this hardware RAID controller support to passthrough the disks directly, e.g. as JBODs?

FYI, if you are in the process of building a cluster with distributed storage and want to use Ceph, Ceph does also not support using hardware controllers, but should have direct access to the individual disks.

[0] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_hardware_2

 

[foolonthehill]

Thankx Daniel,

the RAID Controller supports HBA mode.
We had to disable Hardware Cache for HBA drives and Hardware Cache.
The system run stable now for several days.

Have a great weekend,

Harry

 

[dakralex]

Thanks for getting back and great that the system runs stable now!

Cheers!