Configuration Advice for Proxmox Newbie

[crackerjak]

Hi Proxmox pros...I was hoping some SMEs could give me a hand on some advice on my new Proxmox implementation. I was a former Freenas user, but I felt like it was limiting in key areas that are important to me (i.e. VM hosting and native Plex support).

Hardware

• Dell Precision T7500
• CPU: Dual Xeon x5675 (3.07ghz)
• RAM: 48gb ECC
• HBA: ibm m1015 flashed to it mode
• HDD: 4 x 4TB Seagate NAS
• SSD: 3 x 250GB Samsung EVOs; 1 x 120GB PNY CS1311

Usage/Intent
This server will act as a home server (and homelab) for a 6 people. While the primary VM will be a file server, another big function is to store and serve my media collection (HD Movies, pictures, and music) via Plex to a number of TVs and PCs throughout the house. Additionally, I will be using it as a game hosting server for my kids (i.e. Minecraft, Ark, Teamspeak 3, and any else I want to play around with). While writes speed would be nice, the primary focus is read performance.

What would be a good configuration with the hardware above? I realized the SSDs are all consumer grade, but that's what I have on hand and I'm not doing anything "heavy" or "mission critical" that requires the max amount of uptime or reliability. I keep multiple backups on/off site, so I am willing to accept that I will have SOME level of risk.

What I was thinking

• HDDs as stripped mirrors for data storage
  
• The 3 EVO SSDs in Raidz for VM and LXC storage (Install Proxmox in an rpool? Are there any "gotchas" by doing this)
  
• The PNY SSD either as either root or as a cache for the Raid10 (although I feel the drive is crap for this use and that I may not need/notice a performance increase with an ssd cache with my use case above). Or I can just leave it out all together and repurpose it.

Any advice and/or insights would be greatly appreciated. BTW...sorry for the wall of text

 

[hybrid512]

Well, I would personnaly do this :

• PNY SSD for system installation (I usually give 12GB to my ProxMox nodes with Ext4, it's far enough unless you want to install stuff on the hypervisor itself which I wouldn't recommend). You could use extra space for whatever you want.

Having no raid for the OS itself is not critical since this is for personnal use.
If you dedicate this SSD to your root installation, if it fails, it will be easy to reinstall a new system on another disk and retrieving your current VMs/CTs which are stored on the other volumes.​

• EVO SSDs in RAIDZ volume as you said and create a "fast" local volume to store your i/o hungry volumes/VMs

• RAID10 with your HDD if you need a good mix between performances and security, otherwise, a RAIDZ would be perfect for high capacity storage (compressed RAIDZ + 4x4TB would give you approximately 12 to 16 GB of storage space with a RAID5 equivalent in terms of security but with better performances) but it would be a little slower performance wise.

I wouldn't recommend installing rootfs on ZFS even though it is perfectly possible and working.
I would dedicate my storage (SSDs and HDDs) to local ZFS volumes with a "fast" one and a "big" one that you can mix the way you want based on your needs in your VMs/CTs.

Since you're having only one host, I would also recommend, whenever possible, to use LXC containers instead of VMs except if you need specific kernel feature or different OSes such as windows or others.

Have fun !

Regards.

 

[crackerjak]

Thanks for the reply. I like the idea of storage tiering and I think that would work nicely.

Since rootfs doesn't take up too much space, do you thing I would benefit from partitioning part of the PNY for l2arc on the big/slow array? Since writes are my primary concern, I don't THINK I would benefit much from a zil log. Also, I image the PNY would get clobbered on writes (consumer ssd), but would be fine for a read cache...is this assumption incorrect?

 

[hybrid512]

Well, be carefull with consumer SSDs ... I had big troubles with Corsair LS ones (60Gb for less than 30€ each ... I thought they would do great rootfs disks but they died after a few months even though except for logs, they were not very busy ...).
On the other hand, I use Samsung 850 EVOs (not pro) on a write heavy ZFS storage and they live quite long considering how much write ops they take ... so consumer SSDs might be just ok for your needs but just be carefull with them especially when you know that you can have a low storage Intel DC SSD for just a few more bucks. (I get Intel DS S3520 150GB SSDs for under 100€).

I really don't think using this SSD for ZIL would help considering one of your ZFS volume will be SSDs based and the other one would be more targeted at high capacity storage than performances.

And don't forget to enable ZFS compression ... it really helps in both capacity and i/os limis.

 

[crackerjak]

Thanks for all of your insights...I'll give it a whirl and see it goes.