Storage config guidance for first time Proxmox setup

[the-gloaming]

After a bit of research and liking what I see here, I've come around to going the Proxmox route to setup a home server (vs. UnRAID or a standalone Ubuntu server). This will be a single server for different things which I'd like to setup and play around with and see where it takes me - my own Git server, home automation, running different docker applications, and eventually a NAS server.

I am trying to figure out the right storage config for the same, and my current thinking is as below (and visually at this link: https://imgur.com/NOXEWiu ).
Appreciate your inputs in guiding me to the right direction!

1. Proxmox OS and VM images running on two 128GB SSDs (in HW RAID1 config, based on the motherboard). Potentially ZFS (as that seems to be the Proxmox recommendation?)
2. LVM(s) for storage aspects for a Ubuntu Server VM, LXCs, Docker containers (within or outside the Ubuntu VM / LXCs - tbd), and a Windows VM (potentially). 2 x 2TB HDD or 2 x 4 TB HDD to start with, may expand it later (likely Seagate Ironwolf or Seagate Barracuda). Cant do HW RAID as I am using that for the OS SSDs (or should I use that for the storage HDDs?)
3. [for later] Standalone NAS VM (likely TrueNAS) with 4TB HDDs (Seagate Ironwolf) connected via a HBA passthrough board. Total number of HDDs to be seen (depending on budget); for the same reason, likely to go with SATA drives (as SAS looks way too expensive here). Likely as ZFS (esp. if TrueNAS).

 

[Dunuin]

Using no raid at all or onboard raid is always the cheapest but worst option.

If you just want a fast raid but don't care about advanced features (like deduplication, blocklevel compression, bit rot protection, replication and so on) get a HW raid card with BBU, two Enterprise grade SSDs for a raid1 and use LVM-Thin ontop.
If you don't care that much about performance and SSD life expectation but really care about data integrety or if you want the advanced features you should get two Enterprise SSDs for a ZFS mirror. ECC RAM is always good to have, especially when using ZFS. And in case of ZFS you also need to buy CMR HDDs (but SMR HDDs are in general always a bad idea to buy).

And 128GB sounds a bit small. You want 16-32GB for your PVE, alot of free space in case you want to use snapshots and in case of ZFS only 80% of the capacity should be used. So 128GB will just fit the Win10 VM. In general you don't want to store VMs on the HDDs because they can't handle enough IOPS. They are fine for some cold storage but I wouldn't want to run the guests OS on them.

And make sure you get a decent amount of RAM, especially when using ZFS/TrueNAS. And for TrueNAS you also might want to buy a HBA card so you can use PCI passthrough so your TrueNAS VM can physically/directly access the HDDs without any virtualization/abstraction layer in between or you will get additional overhead and stuff like health monitoring inside TrueNAS won't work.