Adding SSD for cache - two ZFS pools

[nightcom]

Hi all,
I would like to improve performance in my second pool 3x1TB HDD RAIDz1,I want to lower I/O delay its not high but I was reading also that there shouldn't be any I/O delay, (first pool where is Proxmox I have 2xSSD in RAIDz) by adding SSD drive as cache. I found already on Google some information's how to do it
after creating two partitions on SSD (same size?)

zpool add rpool cache /dev/sdf1
zpool add rpool log /dev/sdf2
zpool status

and now I can't find right answer and that's why I came here again for advice
My questions are
1. If I want just for second pool should I write name of second pool instead "rpool" (zpool add secondpoolname cache....)?
2. What will happend when SSD with cache will die? Just performance will drop or some data can be lost also?
3. 2xSSD in Raidz that I'm using for Proxmox, is it good idea to create cache on same SSD in this Raidz and Proxmox on it?

 

[nightcom]

anyone?

 

[wolfgang]

Hi,

If I want just for second pool should I write name of second pool instead "rpool" (zpool add secondpoolname cache....)?

Generall, it makes no sense to add an SSD as cache (ZIL) device to an SSD Pool.
A Cache will only improve the speed if the cache device is faster then the Pool.
Also, the ARC as to be used with care, because it is not for free and will need about 300KB per block in memory.

What will happend when SSD with cache will die? Just performance will drop or some data can be lost also?

For the L2ARC it makes no difference.
And for the ZIL it depends.
1.) only the ZIL is failing and the Server stays running nothing happened because the Data are in Memory.
2.) the Server crashes and the ZIL is dead, you will lose all the data from the not committed Transaction Group. But the FS stays consistent.

2xSSD in Raidz that I'm using for Proxmox, is it good idea to create cache on same SSD in this Raidz and Proxmox on it?

A Raidz aka raid5 makes no sense for two disks.
The cache device has to be a dedicated device if you like to see any performance increase

 

[Tekuno-Kage]

Hi all,
I would like to improve performance in my second pool 3x1TB HDD RAIDz1,I want to lower I/O delay its not high but I was reading also that there shouldn't be any I/O delay, (first pool where is Proxmox I have 2xSSD in RAIDz) by adding SSD drive as cache. I found already on Google some information's how to do it
after creating two partitions on SSD (same size?)

zpool add rpool cache /dev/sdf1
zpool add rpool log /dev/sdf2
zpool status

and now I can't find right answer and that's why I came here again for advice
My questions are
1. If I want just for second pool should I write name of second pool instead "rpool" (zpool add secondpoolname cache....)?
2. What will happend when SSD with cache will die? Just performance will drop or some data can be lost also?
3. 2xSSD in Raidz that I'm using for Proxmox, is it good idea to create cache on same SSD in this Raidz and Proxmox on it?

I used to have similar setup:

1. If I want just for second pool should I write name of second pool instead "rpool" (zpool add secondpoolname cache....)?
You could install Proxmox on a small partition 16GB (ZFS Mirror), use a second partition for RAIDz2 with L2ARC cache (I named: Storage_zPool) and a 3rd partition for Swap (Not part of ZFS) remember divede the total swap size by 2 and that should be the partition size, you will end with two swap partition one on each disk. That will make a busy SSDs.

2. What will happend when SSD with cache will die? Just performance will drop or some data can be lost also? If you fallow the above setup, you will loose a mirror disk on the rpool, half of the cache performance improvement, no data loose (at least never happens to me and I loose cache disk), and half of the swap file (if you don't want to loose the half of the swap, create the partition with the full swap file that you will need).

3. 2xSSD in Raidz that I'm using for Proxmox, is it good idea to create cache on same SSD in this Raidz and Proxmox on it? Not able to comprehend the question.
My recommendation for just zfs environment:

rpool (16GB)
  mirror
    ssd-1p2
    ssd-2p2
Storage_zPool (Around 2TB usable)
  L2ARC
    ssd-1p3
    ssd-2p3
  RAIDz1
    HDD-1p2 
    HDD-2p2
    HDD-3p3

If I remember properly you will end with 5 partition on the SSD. After the install Proxmox create 3 partitions, plus the L2ARC and the Swap you get your disk full allocated.
I like this reference for swap size selection: https://itsfoss.com/swap-size/

Regards and Good Look,

 

[wolfgang]

Hi,

My recommendation for just zfs environment:

it is not recommended to use a ZFS data disk as ZIL for another pool.
This can end in a deadlock.

 

[Tekuno-Kage]

Hi,



it is not recommended to use a ZFS data disk as ZIL for another pool.
This can end in a deadlock.

I agree with you, even so, on the presented scenario the disk are SSD, and the rpool should just be use for running the OS, not VMs/LXC. I forget to mention that I deactivate everything from rpool disk from the Datacenter Storage section. Therefore the load of the SSD disk will be OS transnational info like logs, etc...
Don't forget that I also add the swap, but the same apply, if the system have enough memory and the vm.swappiness = 1, the swap will be use just at last resource.

Documentation need an update (the new version didn't swap on zfs by default), but anyway could be configure as "Minimum amount of swapping without disabling it entirely. "
https://pve.proxmox.com/wiki/ZFS_on_Linux

For the record, if the implementer could find enough $$$ to put everything apart, that would be the best option. My proposed solution is on a constrain budget...