Best ZFS practices using Proxmox install and after

[sahostking]

New Server we want to setup with the following:

64GB ECC RAM
2 x 240GB SSD disks
4 x 2TB disks for data.

New to ZFS and would probably like ZFS caching so we can have it fast as possible.

How should the RAIDs be setup for optimal performance? I usually use HW Raid controllers so this is a bit of change.

Note: We will be hosting a few KVM servers on it.

Thoughts?

 

[mbaldini]

ZFS RAID 1 with the two SSDs, install proxmox on that and use that (local-zfs) for VM disks that need fast storage.
Then ZFS RAID 1+0 with the four spinners, add the new pool as storage on proxmox and use that for VM disks that are ok with a slower storage (small or seldom used datas, and so on)

64GB can be low if you want a good ZFS ARC cache, according to how much RAM will your VMs will be using. If you have free RAM, 16GB or more of ARC cache will be very useful in speeding things up.

 

[sahostking]

In total they use around 36GB on avg.

Basicall the KVM VMs are cpanel VMs.

We currently use pureSSD disks but trying to save costs with our hosts aswell as have more diskspace on the servers.
The Pure SSDs using pveperf is around 500MB/s +

So we thought because disks avg usage is 10MB - 20MB/s it would be fine on SSD caching system to save costs using the 2 x 240GB and 4 x 2TB disks.

 

[Rhinox]

If you can not use pure-SSD setup anymore, it is better to let ZFS decide which data are frequently needed. So personally I'd use 2xSSD just for PVE + zlog + l2arc (+ swap, if you do not have enough RAM), and 4xHDD for everything else.

 

[guletz]

Hi,

The most important best practices is to read the zfs documentation. This is the most important step. The second step is to try in a test environment as many case as you could like: replace a faulty disk, replication, zfs clone promote, snapshots, and many others things.
The 3rd thing is to test some basic settings like arc memory limit, metadata memory limit, zvol performance using different zvol sector size, how to setup zvol for different cases like win/linux os, or in the case of using some SQL engine.

Without this basic steps, is only a matter of time when you will need to go to learn this elementary steps.