Proxmox + ZFS + NAS

macamba

Member
Mar 8, 2011
30
0
6
After long hesitating: Mini-pc and Synology NAS vs. custom build pc with Proxmox and Virtual NAS (probably OpenMediaVault). I will go for the latter option. Just wanted to check whether the setup I am heading for is feasible:
1) I want to install the Proxmox (operating) system to a SSD.
2) 2 or 3 HDD disks I want to use for data based on ZFS file system (RAID1 or RAIDZ1).
3) I want to use SSD caching.
4) Now and then I do a clean reinstall of the Proxmox system the ZFS file system should remain intact.
5) OpenMediaVault will run in a container and folders of host will be local shared with the guest container via bind.

Question, do I need 2 SSD's, i.e. one for Proxmox and one for SSD caching or can I combine one? Taking into account I want to keep stuff as independent building block as possible. Is the SSD cache drive integral part of the ZFS pool? Because if it is it might me handier to have separate ones for the OS and caching.
 
Last edited:

aaron

Proxmox Staff Member
Staff member
Jun 3, 2019
198
10
18
In theory you could partition the SSD to use one part for the OS and the other for the L2ARC (cache disk).
But our installer cannot handle this. It will always use the full disk.

You did not post how much RAM you intend to use and how big the disks are going to be. Be aware that the L2ARC also costs you RAM, reducing the amount of RAM the ARC (ZFS cache in RAM) can use. Many times the money for the cache SSD is better invested in more RAM. Especially if you are at the lower end already.
 

macamba

Member
Mar 8, 2011
30
0
6
In theory you could partition the SSD to use one part for the OS and the other for the L2ARC (cache disk).
But our installer cannot handle this. It will always use the full disk.

You did not post how much RAM you intend to use and how big the disks are going to be. Be aware that the L2ARC also costs you RAM, reducing the amount of RAM the ARC (ZFS cache in RAM) can use. Many times the money for the cache SSD is better invested in more RAM. Especially if you are at the lower end already.
I read somewhere that 8 GB of RAM is recommended, that's basically the guidance I intended to follow.
 

aaron

Proxmox Staff Member
Staff member
Jun 3, 2019
198
10
18
I read somewhere that 8 GB of RAM is recommended, that's basically the guidance I intended to follow.
Depending on how much RAM your VMs need you will need more RAM. I found this old thread [0] discussing the L2ARC and people reported that it wasn't optimal.

If you still want to go down that route try and see how the performance differs with L2ARC enabled and without it. Give it enough time to warm up and get filled though.

[0]: https://forum.proxmox.com/threads/zfs-l2arc-sizing-and-memory-requirements.23601/
 

guletz

Well-Known Member
Apr 19, 2017
1,016
144
63
Brasov, Romania
Hi,

l2arc is useful if yours clients will access the same data most of the time. If you do not have enough RAM, l2arc will reduce overall performance, as @aaron said.

And, any general propose cache (like l2arc) is less effective compared with guest OS internal cache(is more closely to the clients compared with l2arc).

So a better performance could be obtained like this:

- use l2arc only for metadata
- add more RAM to your guests(better cache hit rate then l2arc)

Good luck
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!