Server Design for Proxmox w/ ZFS

Steve88

New Member
Apr 12, 2019
4
0
1
35
Hi,

I am currently configuring a server with the goal of running Proxmox for several Windows 10 VMs and a Linux Server. The total amount of disk space should be approx. 4 TB.

My current favorite design would be Raid-Z2 with 6 x 1TB SAS HDD for the VMs. Now I am struggeling a bit with ZFS caching and Proxmox installation. My initial idea was to install proxmox on a DOM and use 1 SSD for L2ARC and 2 SSDs in Raid1 for ZIL. Then I heard that it is probably a bad idea to install Proxmox on a DOM for endurance reasons. Would it be possible to install it on the mirrored SSDs in a separate partition (on top of Debian in this case) and would there be performance implications? Would a separate SSD be the better solution? Or is there any other good option I am not considering right now?

Many thanks in advance!
Cheers
 
Last edited:
Hi Roman, no, sorry I meant disk space (just corrected my post). The amount of available RAM should be 128 GB in total.

Actually I have little clue about CEPH. But as you suggested it, I will take a look into it.

Anyway, are there also any suggestion about a good design for ZFS usage?
 
Hi Steve,
it depends on the requirements to be given.
Should ceph come into question as storage solution, there must be at least three nodes (servers).
Right now in the fresh version 5.4 it is possible to completely configure ceph via the interface. ceph has almost only advantages, here should also no RAID for the ceph hard drives disks - except with the operating system Proxmox.
If you "only" have one server for proxmox, ZFS with SSD-Disk works very fast on a local node.

Best regards, roman
 
Many thanks for the information! Then I guess I will stay with ZFS as I will only have this one server.

So for ZFS, what do you think about reserving 32 GB ARC and a 128 GB mirrored ZIL as many sync writes are expected. Proxmox on a separate SSD. Would an L2ARC make sense in this config?
 
you dont need that much ZIL; consider underprovisioning your SSDs so at least they live longer. Have a look here: https://www.ixsystems.com/blog/o-slog-not-slog-best-configure-zfs-intent-log/

With current versions of zfs, slog failure is not fatal; you may consider not bothering with the mirror depending on your tolerances.
Would an L2ARC make sense in this config?
Not usually, although some use cases can see benefit depending on amount of RAM in the system. In a nutshell, when using l2arc A large fraction of the l2arc size is required in RAM for metadata so the end result could be slower (or not meaningfully faster) if you just left ARC in RAM. Search the interwebs for benchmarks most resembling your load type, but generically I'd say dont bother.
 
  • Like
Reactions: HE_Cole
Many thanks for the link and the information! I guess I'll go for a little SSD for ZIL and some additional RAM for ARC instead of L2ARC.

Thank you so much for your help! :)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!