Need your opinion what raid and file system format to use for ProxmoxV4 and Cache

[bitblue.lab]

I will migrate to a new server with 2x 2TB Hard drives and 2 x 300GB SSD Drives and with 64 GB RAM ECC so I want to use 2x2TB Drives as Raid 1 and one of the SSD drives as cache for the raid 1 config and other one as a single disk.

1.Is it better to use ZFS File System for Raid 1 2x2TB Drives and how to add one ssd of 300GB as cache to this raid ?
2.Is ssd going to help 2x2TB Raid 1 Config ?
3.Can this system failure in data read-write in case of any power outgage or any reboot ?

Waiting for your opinion.

 

[LnxBil]

Why not use the two 2TB Disks for ZFS for install and then use the two remaining SSD for L2ARC and mirrored ZIL?

https://pve.proxmox.com/wiki/Storage:_ZFS#Create_a_new_pool_with_Cache_.28L2ARC.29

Then you have fault-tolerance and performance exactly as you wanted.

 

[bitblue.lab]

I wanted to use only one of the disks ssd as cache, because the other SSD I need for other purposes...or its a must to have log/zil ? In case of failure can the system repair and work again without mirrored zil ?

 

[LnxBil]

You can only have a read cache if you only want to use one SSD. Of course, if you want to have no single point of failure - if you want to have a zil on only one SSD you will have data corruption if the SSD breaks.

Why not use the SSD in a mirrored setup and use space to your other, not mentioned use case? Using 300 GB of L2ARC is really a lot and you need a lot of RAM to manage the L2ARC.

 

[bitblue.lab]

You can only have a read cache if you only want to use one SSD. Of course, if you want to have no single point of failure - if you want to have a zil on only one SSD you will have data corruption if the SSD breaks.

Why not use the SSD in a mirrored setup and use space to your other, not mentioned use case? Using 300 GB of L2ARC is really a lot and you need a lot of RAM to manage the L2ARC.

Can I use same SSD for read and write ? I dont want to use ssd-s in mirrored mode both of them because I can't use them for anything other only as cache.

So my plan is to use same SSD for read and write cache because its 300GB SSD with read and write up to 550MB/s .

What do you think about that ?

 

[LnxBil]

Sorry, but if you want to build a single point of failure, go ahead. You cannot use only one device for secure write-back cache. It is just NOT possible. The data is written just ONCE.

Without knowing your workload, 550 MB/sec is not the right kind of spec for buying an SSD. You buy an SSD for IOPS, not for throughput (MB/sec). You have to check if this SSD is good for ZFS ZIL use. You need good IOPS performance, not throughput. If you use a "slow" SSD, you will not gain the power you expect from adding a SSD to your ZFS.

My suggestion is to partition your SSD to have a mirrored-zil (e.g. 16 or 32 GB) and 64 GB of each (non-mirrored) for L2ARC. The rest you can use in an mirrored setup for your other purpose, because you need mirroring everywhere if you want to handle data correctly.

 

[bitblue.lab]

So the ZIL is the write cache part, and L2ARC is the read part.

My SSD-s have 75k iops read and 11k write...I understand now better my mirroring configuration I will put 2x2TB Drives in mirror then use from both SSD-s one part from them for RAID 1 mirrored around 50-100GB, then only from one one part for using read cache, and other part left from SSD-s I can use for other things that I want.

 

[LnxBil]

50-100 GB for ZIL is IMHO too much. Please consider the following article:

https://www.ixsystems.com/whats-new/why-zil-size-matters-or-doesnt/

 

[bitblue.lab]

50-100 GB for ZIL is IMHO too much. Please consider the following article:

https://www.ixsystems.com/whats-new/why-zil-size-matters-or-doesnt/

I will play with ZIL later and tune it...currently I have another problem. When I create new hard drives in proxmox for vm-s it creates new disks in system like /dev/sdb etc with names zd01,zd16 etc...Can someone help me on this I am migrating to new infrastructure...I need files of VM-s etc to be saved in the directly of my zfs pool /storagepool. At the moment for every disk created in VM Images in Proxmox its created a new disk on system and somehow is using ZFS but not letting me able to see the data etc.