Hi all, I am configuring my homelab/NAS device and need a sanity check and some advice. This is my current HW setup
I wish I could separate the NAS into its own thing, but because of space and noise and budget constraints I really can't. I also don't have a lot of flexibility in changing the hardware.
Since all the PCIe x16/x8/x4 slots are taken up by my GPU, I also don't have any to spare for HBAs. I have a couple PCIe 4.0 x1 slots available for expansion if necessary.
Now my questions are:
Any other general advice/tips would be greatly appreciated. I know its a lot and I would be very grateful for any inputs.
P.S: I'm an ML Engineer with plenty of terminal experience so I'm not worried about my hands dirty, but still a noob in networking/virtualisation. I would like to (try to) setup everything as optimally as I can from the beginning so as to not have too many headaches in the future.
- Motherboard: MSI X570S PG Riptide
- CPU: AMD 5700X
- SSD: 2TB NVME on M2_1
- RAM: 48GB 2666Mhz ECC
- HDD: 4x Seagate 10TB directly connected to the SATA controller
- GPU: 2x RTX A4000
- NIC: Intel x520-DA2
I wish I could separate the NAS into its own thing, but because of space and noise and budget constraints I really can't. I also don't have a lot of flexibility in changing the hardware.
Since all the PCIe x16/x8/x4 slots are taken up by my GPU, I also don't have any to spare for HBAs. I have a couple PCIe 4.0 x1 slots available for expansion if necessary.
Now my questions are:
1. I'll be passing the entire SATA controller to the TrueNAS VM. So do I partition the NVMe and use it for installing Proxmox and TrueNAS?
2. How much RAM should I allocate to the VMs? Is it possible to do dynamic allocation?
3. Should I use NFS or SMB for accessing data on the TrueNAS VM from my Proxmox VMs/containers. I read about how NFS uses synchronous writes and would be slower. Would SMB be better in that case. Do I need to get a separate SSD for SLOG if I use NFS? What speeds could I get, in theory, in either case?
4. I'm planning to use RAIDZ2 on my pool. In the future if I wish to expand my storage, would be possible to add another vdev with just 2 more HDDs in mirror mode? I do not plan to expand beyond that for now as I don't have such high data needs. Even 20TB is way too much for me (famous last words).
5. At some point I plan to add two more SSDs in mirror mode to act as "special devices" that would be used for storing metadata which in theory should make my pool run faster. But since I'm out of ports, is it a good idea to use a PCIe x1 to M.2 adapter like this (https://www.amazon.com/GLOTRENDS-Adapter-Installation-Bandwidth-PA09-X1/dp/B09P3HY3P3)
6. If I plan to do some video editing directly from the TrueNAS on my Macbook Pro, what's a good idea to increase read speeds? This is not something that I would be doing frequently so not a high priority. Except for increasing RAM would reserving some space on the NVMe for L2ARC help?
7. How can I back up the Proxmox installation and settings? Incase the NVMe fails or something. Is it possible to back it up inside the TrueNAS VM. How I recover the TrueNAS VM/Proxmox VE in that case?
8. Adding to the previous question, can I back up the other VMs/Containers running on the Proxmox VE, inside the TrueNAS VM?
Any other general advice/tips would be greatly appreciated. I know its a lot and I would be very grateful for any inputs.
P.S: I'm an ML Engineer with plenty of terminal experience so I'm not worried about my hands dirty, but still a noob in networking/virtualisation. I would like to (try to) setup everything as optimally as I can from the beginning so as to not have too many headaches in the future.