Looking for best setup for rack server

[str1de]

Hello all! New to Proxmox, so forgive my ignorance. I'm looking to have my server sent for colocation and will have proxmox installed for multiple VM's. My main goal will be for small vm's for linux, small game servers, or just general usage. My issue is that I have 8 total enterprise SSD'S. I have 4 (480GB) and 4 (240GB) in total. I read where ZFS might be a good idea to make them all one big array, but I also heard that it takes up a ton of ram for caching which isn't ideal at the same time. I'm looking for suggestions / help to make sure I do this right. Thanks for any help in advance!

My server is older but not too bad for a starting piece of hardware. My server is:

SR550 (Lenovo ThinkSystem) - Type 7X04
Dual Xeon Silver CPU 4110 2.0GHZ ( 16 cores 32 threads)
96GB DDR4 RAM.

 

[alma21]

Hi,
ZFS memory consumption is configurable and Proxmox VE takes 10% of total memory for ARC cache per default .... so this should not be a problem .... if no HW Raid Controller exists then ZFS is the only supported SW RAID solution ... 2 small drives with ZFS mirror for Proxmox OS and the remaining 6 SDDs in a ZRAID5/6 or ZRAID10 config vor VMs/data

 

[str1de]

Thanks for the reply! I also contemplated putting in a m.2 drive via PCIE 3.0 card to strictly hold proxmox so I don't touch the 8 drives at all. Would that still be a good idea? Something along these lines PA09-X1 M.2 NVMe to PCIe 4.0 X1 Adapter

Hi,
ZFS memory consumption is configurable and Proxmox VE takes 10% of total memory for ARC cache per default .... so this should not be a problem .... if no HW Raid Controller exists then ZFS is the only supported SW RAID solution ... 2 small drives with ZFS mirror for Proxmox OS and the remaining 6 SDDs in a ZRAID5/6 or ZRAID10 config vor VMs/data

 

[alma21]

Thanks for the reply! I also contemplated putting in a m.2 drive via PCIE 3.0 card to strictly hold proxmox so I don't touch the 8 drives at all. Would that still be a good idea? Something along these lines PA09-X1 M.2 NVMe to PCIe 4.0 X1 Adapter

possibilities:
- 1 drive for PVE OS only -> if this drive fails you should have a good backup .... then I would not use ZFS for the OS - go with LVM/ext4 for OS setup
- ZFS features like bitrot detection & correction work only with >= 2 drives in some mirror or ZRAID config ... ideally with enterprise SSDs

 

[UdoB]

Thanks for the reply! I also contemplated putting in a m.2 drive via PCIE 3.0 card to strictly hold proxmox so I don't touch the 8 drives at all. Would that still be a good idea?

Well, "a m.2" means a single one? That's a bad idea ;-)

If you can install two (or more) of them the following sentence would be completely different:

While it is still best practice to separate "the OS" from "the data" it is absolutely not a requirement. If you just install on a striped mirror (aka "Raid10") then the result is a single "rpool". This one is good for a) the OS, b) block devices for VMs and c) data sets for Containers and d) other "normal" stuff.

The benefit is that at the end each and every physical device is bootable.