I am building a homelab on a Qotom Q20331G9. System board has two onboard PCIe 3.0 x2 M.2 and two onboard SATA plus a SFF8087 port. I want to play with a virtualized instance of TrueNAS and would eventually like to configure it with SSD caching. I am looking for opinions on how to best configure the system.
Should I put Prox OS + VMs on NVMe and using SATA SSDs for TrueNAS metadata/cache. Or should I put OS + VMs on SATA and use the NVMe for TrueNAS? Initially I will have just two SSDs to present to TrueNAS in a simple two-drive pool but once I add a 4-bay SFF8088 drive chassis I plan to build a pool from four SATA spinning disks and repurpose the SSDs as cache drives.
I've also considered the possibility that the 4 SATA ports provided via the SFF8087 are on the same controller as the two onboard SATA so that would make the decision for me, since I want to pass the entire controller to TrueNAS I won't have any choice but to put Proxmox OS + VM on the NVMe. I haven't investigated that yet though.
Assuming I can go with either configuration, would I see a noticeable performance advantage using the NVMe for TrueNAS even if that means running Prox (and more importantly the VMs) from the SATA SSD? Or should I give my best disks to Proxmox VMs and let TrueNAS cache on the SATA SSDs? I do plan to run other virtual machines but this is only a 8-core Atom CPU so it'll remain a relatively light load. Also, while they are about 3x faster than a SATA interface, these are only gen3 PCIe and only x2 at that, so they're only a quarter the speed of PCIe 4.0 x4
Something else to consider, my plan was to run a 2-drive mirror ZFS volume for the Proxmox OS and rpool (either way, with NVMe or SATA SSD) but if that is a poor design choice given my limited I/O device options should I do something different with these drives? Maybe OS on SATA SSD, VMs on one NVMe, pass the other NVMe to TrueNAS? This is just a homelab but I do eventually plan to migrate router duties for my home network to a virtual instance of VyOS or OPNsense so I kinda want the redundancy of a ZFS mirror volume for OS and VMs.
Thanks in advance for your feedback/opinions.
Should I put Prox OS + VMs on NVMe and using SATA SSDs for TrueNAS metadata/cache. Or should I put OS + VMs on SATA and use the NVMe for TrueNAS? Initially I will have just two SSDs to present to TrueNAS in a simple two-drive pool but once I add a 4-bay SFF8088 drive chassis I plan to build a pool from four SATA spinning disks and repurpose the SSDs as cache drives.
I've also considered the possibility that the 4 SATA ports provided via the SFF8087 are on the same controller as the two onboard SATA so that would make the decision for me, since I want to pass the entire controller to TrueNAS I won't have any choice but to put Proxmox OS + VM on the NVMe. I haven't investigated that yet though.
Assuming I can go with either configuration, would I see a noticeable performance advantage using the NVMe for TrueNAS even if that means running Prox (and more importantly the VMs) from the SATA SSD? Or should I give my best disks to Proxmox VMs and let TrueNAS cache on the SATA SSDs? I do plan to run other virtual machines but this is only a 8-core Atom CPU so it'll remain a relatively light load. Also, while they are about 3x faster than a SATA interface, these are only gen3 PCIe and only x2 at that, so they're only a quarter the speed of PCIe 4.0 x4
Something else to consider, my plan was to run a 2-drive mirror ZFS volume for the Proxmox OS and rpool (either way, with NVMe or SATA SSD) but if that is a poor design choice given my limited I/O device options should I do something different with these drives? Maybe OS on SATA SSD, VMs on one NVMe, pass the other NVMe to TrueNAS? This is just a homelab but I do eventually plan to migrate router duties for my home network to a virtual instance of VyOS or OPNsense so I kinda want the redundancy of a ZFS mirror volume for OS and VMs.
Thanks in advance for your feedback/opinions.