Slow ZFS encryption inside TrueNAS Scale

[mathiasdev]

I'm running TrueNAS SCALE ElectricEel 24.10 inside Proxmox VE 8.3.3 and the disk access is extremely slow when using encryption on datasets. I understand that encryption will hit the disk performance, but the speed is so low that it's not usable and I believe something might be wrong in my settings.

In TrueNAS I have one pool of two mirrored 8 TB enterprise drives connected via a LSI 9211-8I SAS card using passthrough.

Inside my Pool:

Dataset 1: no encryption - read/write speed >280 MB/s (only limited by my 2.5G network)
Dataset 2: encryption AES-256-GCM - read/write speed ~30 MB (sometimes dropping way below)

My VM is using following settings:

Memory: 16 GB
Processors: 4(1 sockets, 4 cores) [x86-64-v2-AES, flags=+aes]
BIOS: OVMF (UEFI)
Machine: q35
SCSI Controller: VirtIO SCSI single
PCI Device (hostpci0): mapping=SAS-controller

My best guess is that the q35 is not actually using hardware acceleration for AES-NI instructions.

Any thoughts?

 

[IronWisp]

What CPU does the machine have? Older xeons can cause bottlenecks while encrypting data
Are you using disk cache?

 

[mathiasdev]

I'm using an Intel i5 12400 (6C, 12T) and no additional disk cache. I also have another server running TrueNAS Scale on bare metal with the same encryption enabled which does not seem to affect the performance much at all.