Storage noob here. I am building a new single node proxmox server on a 2U server that has 9 3.5in 4TB HDDs and 6 2.5in 800GB SSDs beyond what is used for Proxmox's boot image. The server will be running a mix of stateful container workloads, databases for stateless containers, and VMs for traditional desktops and servers. I have been trying to figure out what makes the most sense for the storage setup considering redundancy and optimizing the different types of workloads for performance.
Here's where my speculation begins, please correct any misunderstandings I have.
For the VM images, I am thinking about doing LVM-thin on hardware RAID with raw disk images for a balance of performance and snapshot capability. For the VM images, placing them on SSD vs HDD will depend on the size and use case of the VM.
For the container workload storage I am thinking about passing disks through to a TrueNAS Scale VM and having that handle a ZFS setup with vdevs for both the HDDs and the SSDs. The reason I am thinking TrueNAS VM is because it seems easier to set up and connect to as the storage for my Kubernetes workloads. It seems like I will want to store the databases on the SSDs and most of the other stateful workloads and mass storage on the HDDs.
I've also looked a little into Ceph as the storage backend, but I'm not sure if I fully understand the benefits and drawbacks of Ceph or if that even makes sense on a single node server like this. Some thoughts on this and how it compares to my previous ideas would be appreciated.
I'd love to hear others thoughts on how to handle this.
Here's where my speculation begins, please correct any misunderstandings I have.
For the VM images, I am thinking about doing LVM-thin on hardware RAID with raw disk images for a balance of performance and snapshot capability. For the VM images, placing them on SSD vs HDD will depend on the size and use case of the VM.
For the container workload storage I am thinking about passing disks through to a TrueNAS Scale VM and having that handle a ZFS setup with vdevs for both the HDDs and the SSDs. The reason I am thinking TrueNAS VM is because it seems easier to set up and connect to as the storage for my Kubernetes workloads. It seems like I will want to store the databases on the SSDs and most of the other stateful workloads and mass storage on the HDDs.
I've also looked a little into Ceph as the storage backend, but I'm not sure if I fully understand the benefits and drawbacks of Ceph or if that even makes sense on a single node server like this. Some thoughts on this and how it compares to my previous ideas would be appreciated.
I'd love to hear others thoughts on how to handle this.