ZFS configuration for new build, question about SLOG/L2ARC/Special

[robertklep]

(ZFS newbie, I might be using wrong terminology, please correct me if I do )

I'm going to migrate my Unraid server to Proxmox. With regard to storage I could use some help.

The hardware:

• 2x NVMe 1TB SSD's (Seagate Firecuda)
• 2x SATA 18TB HDD's (Seagate Exos)

Both the SSD's and the HDD's will be mirrored (so a mirror with 2x SSD and a mirror with 2x HDD). I want to use the SSD mirror as boot/root device and HDD mirror for mass storage. I would also like to (eventually) use the SSD for the HDD mirror's SLOG/L2ARC/Special device.

My question: do SLOG/L2ARC/Special require a separate partition, or can they also use a zvol?

In other words: can I let Proxmox set up a mirrored pool on the SSD's using its full size (1TB) and then later create volumes from that pool to use for SLOG/L2ARC/Special, or do I need to partition the SSD drives up front (say 500GB boot/root, then the rest for SLOG/L2ARC/Special partitions later on)?

 

[robertklep]

To answer my own questions: it looks like I can use a dataset for SLOG, instead of requiring a separate partition.

What I did in a VM:

• Install Ubuntu Desktop on a virtual disk (disk A) with ZFS. This creates two pools (bpool and rpool, for boot and root respectively)
• Add a new pool "tank" based on another virtual disk (disk B):

  sudo zpool create tank vdc

• Create a volume on the root pool:

  sudo zfs create -v -V 10G -p rpool/SPECIAL/slog

• Use that volume as log device for the "tank" pool:

  sudo zpool add tank log zvol/rpool/SPECIAL/slog

This seems to have worked:

$ sudo zpool status tank
  pool: tank
 state: ONLINE
config:

    NAME        STATE     READ WRITE CKSUM
    tank        ONLINE       0     0     0
      vdc       ONLINE       0     0     0
    logs
      slog      ONLINE       0     0     0

errors: No known data errors

I'm happy to hear comments on why I should or shouldn't do this.

 

[Dunuin]

My question: do SLOG/L2ARC/Special require a separate partition, or can they also use a zvol?

Separate partition. And SLOG + L2ARC are usually not needed and could make things even worse. But a mirrored special vdev would be a good idea.
Also keep in mind that it isn't recommended to use ZFS with consumer SSDs like your firecudas. Especially when used as a SLOG this will wear your SSDs very fast. And when using the SSD as special devices losing those SSDs also means all data on the HDDs would be lost. So bad idea to put the SLOG on a SSD that contains data you actually care about.

In other words: can I let Proxmox set up a mirrored pool on the SSD's using its full size (1TB) and then later create volumes from that pool to use for SLOG/L2ARC/Special, or do I need to partition the SSD drives up front (say 500GB boot/root, then the rest for SLOG/L2ARC/Special partitions later on)?

No, you tell the installer in the "advanced" menu to keeo some space unallocated so you could later partition that space manually.

To answer my own questions: it looks like I can use a dataset for SLOG, instead of requiring a separate partition.

Bad idea, especially for the SLOG.
You don't want nested ZFS because the overhead will multiply.

 

[robertklep]

Thanks for your comments

I've read up on the pitfalls, it's more a matter of future proofing my new setup so if I ever decide to experiment with using the SSD's I can do so without repartitioning. Based on your suggestions, I'll just leave a chunk of free space on the drives.