RAM for ZFS

[Alessandro 123]

Hi,
I used to have XenServer with hardware raid and 64GB RAM (about 2-4GB for Dom0, the rest for VMs).

By moving to PVE with ZFS, how much RAM should I consider in addition for ZFS ?

 

[mir]

You should count on having 8-16 GB RAM reserved for ZFS.

 

[Alessandro 123]

Good.
This would be our base system: https://www.supermicro.nl/products/system/1U/1028/SYS-1028U-TNR4T_.cfm?parts=SHOW#jump
do you suggest using the additional HBA or go straight with integrated SATA ports ? A missing card will never broke, this is a big advantage but I don't know if performance would be affected.

 

[Alessandro 123]

8-16GB: when 8 and when 16 ? I'll use only SSD, so the L2ARC or ZIL doesn't make sense.

 

[mir]

Good.
This would be our base system: https://www.supermicro.nl/products/system/1U/1028/SYS-1028U-TNR4T_.cfm?parts=SHOW#jump
do you suggest using the additional HBA or go straight with integrated SATA ports ? A missing card will never broke, this is a big advantage but I don't know if performance would be affected.

If you have a HBA I would use it but if you don't I would use the on-board SATA connectors.

 

[mir]

8-16GB: when 8 and when 16 ? I'll use only SSD, so the L2ARC or ZIL doesn't make sense.

Use whatever combination of possible RAM which leaves between 8-16 GB RAM available for ZFS.

 

[gcakici]

AOC-S3008L-L8e is the right card to go with a HBA. It supports IT mode. Don't forget to order extra cables for the back plane connection.

 

[Alessandro 123]

I don't have an HBA, I have to buy it. Is that mandatory or suggested?

About the RAM: why ZFS needs from 8 to 16GB of RAM ? I'm curious, that's a huge amount of memory. Somewhere I've read that 1GB for each TB is suggested.

 

[Alessandro 123]

AOC-S3008L-L8e is the right card to go with a HBA. It supports IT mode. Don't forget to order extra cables for the back plane connection.

Yes, that's the card i'm referring to, but is that needed?

 

[gcakici]

No you don't have to. It gives you SAS ports.

Just go with the existing SATA ports as mir said above.

 

[gcakici]

I don't have an HBA, I have to buy it. Is that mandatory or suggested?

About the RAM: why ZFS needs from 8 to 16GB of RAM ? I'm curious, that's a huge amount of memory. Somewhere I've read that 1GB for each TB is suggested.

If i remember correctly this ram/disk ratio is for CEPH storage calculation.

 

[Alessandro 123]

I'll not use Ceph, but local storage and in the long-term i'll use Gluster with dedicated hosts.