Partitions for ZFS cache and log

[philled]

I'm setting up a new PVE 5.0 server. It has two hard disks (mechanical not SSDs). I want to use ZFS because I believe that's the way to get storage replication from local (as opposed to shared) storage. The installation CD gives the option of ZFS RAID0 and when I choose that both disks are automatically partitioned ZFS.

But I've seen posts advising that ZFS is very memory hungry so it needs cache and log partitions. How can I create these 2 partitions? The installation process doesn't seem to give me the option of setting aside part of a disk for this. It just goes ahead and sets up a standard partition scheme.

Or is it pointless setting up ZFS cache and log partitions on a mechanical disk anyway?

I'm new to ZFS so any advice here would be appreciated.

 

[wolfgang]

But I've seen posts advising that ZFS is very memory hungry so it needs cache and log partitions.

Yes ZFS need memory and no if you use L2ARC you need even more memory.

You can add and remove ZIL (log) and L2ARC (cache) see the reference Doku
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_zfs_on_linux

Or is it pointless setting up ZFS cache and log partitions on a mechanical disk anyway?

You need fast ssd or nvme ssd to profit from this feature.

 

[philled]

Thanks for the reply. I'm noticing that when using dd to move raw disk images onto the Proxmox server, the process hangs quite a lot and it takes ages to dd a 12GB file across a Gbit LAN. The data transfer rate as reported by pv sits at 0 for ages and then shoots up to 34MiB/s then back down to 0 again:

# dd if=/mnt/phill/Backup/VirtualMachines/Ash-Ubu-12.04-64bit/Ash-Ubu-12.04-64bit-2017-09-23_03-17/Ash-Ubu-12.04-64bit.raw bs=1M | pv | dd of=/dev/rpool/data/vm-102-disk-1 bs=1M

5.51GiB 0:15:11 [ 0 B/s]...

I'm wondering if this is because I'm using mechanical HDDs instead of SSDs and I only have 8GB of RAM as this is just a test server, not the real one. Is it a mistake to use ZFS without any SSDs in the mix?

 

[wolfgang]

If you write not a real server do you mean VM?

Spinning disk are fine in the way that are spinning disks.
What I mean is you can't expect extrem speed, but it dependence also on the workload.

 

[philled]

I m

If you write not a real server do you mean VM?

I meant this is a test PVE server, not the real production PVE server. Workloads will be low with approx 5 VMs running.

 

[wolfgang]

To be serious 8 GB is a bit few Memory.
ZFS will only allocate 50% of the memory default.

If you only copy this Ubuntu image and the storage are empty, 4 GB for ZFS should be enough.
But keep in mind this small setups are not made for ZFS and you can run in a corner case where ZFS behave not as it should.

Do you use a Raid card on this test Server?

 

[philled]

To be serious 8 GB is a bit few Memory.
ZFS will only allocate 50% of the memory default.

Do you use a Raid card on this test Server?

Yes, I agree if it's very low. I'll buy some more RAM for it even though it's just the test server.

I don't have a RAID card.