Does L2arc make sense for vm disks?

rockzhou8

Member
Nov 20, 2020
6
2
8
36
I've set up a zfs pool on my Proxmox VE home-server.
sdc and sdd are 2x 4TB SATA HDD
sda is a 480G SATA SSD

NAME STATE READ WRITE CKSUM pool ONLINE 0 0 0 sdc ONLINE 0 0 0 sdd ONLINE 0 0 0 cache sda ONLINE 0 0 0

And there are 3 VM Disks on the ZFS pool .
pve-vm--100--disk--0.raw
pve-vm--101--disk--0.raw
pve-vm--102--disk--0.raw
Each vm disk size are 1 TB.

Here is the iostat when these 3 VMs are running.

avg-cpu: %user %nice %system %iowait %steal %idle 29.65 0.00 19.06 11.99 0.00 39.31 Device r/s w/s rkB/s wkB/s rrqm/s wrqm/s %rrqm %wrqm r_await w_await aqu-sz rareq-sz wareq-sz svctm %util sda 355.90 13.30 3896.10 1138.15 0.10 0.00 0.03 0.00 0.66 0.58 0.02 10.95 85.58 0.31 11.60 sdd 24.80 19.10 968.00 1104.00 0.10 0.00 0.40 0.00 8.33 0.94 0.19 39.03 57.80 2.97 13.04 sdc 152.30 16.10 4433.20 876.00 0.30 0.10 0.20 0.62 20.02 1.68 2.82 29.11 54.41 3.06 51.60


sdc %util is 51.60 and iowait is 11.99 ,So I'm wondering that does l2ARC make sense for these vm disks? Or if I need a nvme ssd for cache instead of the SATA ssd.
Any sugguestions? Thanks!
Sorry for my poor English.
 
Usually a L2ARC or SLOG won't help much. Its more useful to use two (mirrored) SSDs as special metadata devices and to buy more RAM so you got a bigger ARC to speed up the HDDs. But you really want redundant SSDs as loosing the SSD also means loosing all data on the HDDs. 16GB SSDs should be fine for 8TBs of HDDs.

SLOG = only speeds up sync writes
L2ARC = only speeds up reads (and is slower than RAM, so adding more RAM would make more sense)
special metadata device = speeds up reads + sync writes (but not as much as a SLOG) + async writes
 
Last edited:
  • Like
Reactions: rockzhou8
Usually a L2ARC or SLOG won't help much. Its more useful to use two (mirrored) SSDs as special metadata devices and to buy more RAM so you got a bigger ARC to speed up the HDDs. But you really want redundant SSDs as loosing the SSD also means loosing all data on the HDDs. 16GB SSDs should be fine for 8TBs of HDDs.

SLOG = only speeds up sync writes
L2ARC = only speeds up reads (and is slower than RAM, so adding more RAM would make more sense)
special metadata device = speeds up reads + sync writes (but not as much as a SLOG) + async writes
thanks for your advice!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!