Hello, new proxmox (home) user here looking to migrate my old cobbled-together server to something more resilient. This kind of question probably comes up a lot, but I've researched for a week and can't quite put my finger on an answer. Sorry for the length; I though it better to give the background than to provide too little info.
My old motherboard can't support enough RAM for ZFS so after a couple of months of research I've bought a second-hand Supermicro server (x10srh-cln4f, E5-2620 v3 [upgradeable as necessary], 32Gb RAM [upgradeable as necessary]). The use-case for my outgoing server was as a NAS and services for my household - Owncloud, Emby (will swap to Jellyfin), torrenting, JIRA, etc. I want to move to Proxmox for ZFS and the freedom granted by a hypervisor (and of course because I think it's really cool). Single node scenario.
The motherboard has a built-in PCI LSI 3008 controller (which I've already flashed to IT mode), and my intention is to pass this through to a FreeNAS VM, along with around 16Gb RAM to share my spinning rust. As I say I've spent a lot of time reading up on this so I'm happy with this part of the system.
In terms of storage space aside from the NAS, I don't think I need too much for my services, as my household data (Linux ISOs and documents) will be on the NAS. Torrent downloads will go straight to the NAS. I plan to experiment with a few minimal VMs and will opt for containers where I can. I thought to experiment with Docker Swarm (I've loved reading the recipes on https://geek-cookbook.funkypenguin.co.nz/ha-docker-swarm/nodes/), and this means at least 60Gb assigned to VMs (perhaps alleviated by shared storage?). I want some shared storage for VMs (my reading leads me to sharing this from the host as NFS), and I want any databases to be on SSDs (though the databases will be small, I think - no mass-storage, just what the services need). I want to consolidate databases by type, so a single InfluxDB database, a single MySQL database, etc, available to all relevant services.
What I'm not sure about is the sizes of the shared storage for VMs and booting etc. For the non-NAS side of things I have 2x 3.5 bays remaining in my chassis, sufficient for up to 4x SSD drives. My preferences, which I'm willing to be swayed on, are that the boot drive is a mirrored pair of SSDs, and the shared storage is a mirrored pair of SSDs also. But I can't figure out the sizes necessary.
I'm looking to spend £100, plus or minus about 20%. I'm looking at SSDs at prices such as £18 for 120Gb, £28 for 240Gb, and £46 for 480Gb. I couldn't, for example, afford 4x 480Gb.
Should I get, for example, 2x 120Gb (mirrored) for the boot drive, and 2x 480Gb (mirrored) for the shared storage? Is 120Gb enough for the boot drive? I noticed in a test install that Proxmos partitioned a disk into 100Gb for installation, and the rest for <I'm not sure>.
Or would it be best to use the 480Gb mirror for boot + VMs, and 120Gb mirror for shared storage? I have a feeling that 120Gb would be enough for shared storage, but I'm unsure if I can host VMs on the boot drive.
Alternatively, would it be better to back-share some rust from the NAS for some sort of VM storage? I know that VMs (and certainly databases) will perform better on SSDs than HDDs, but I'm not sure what parts of proxmox are moving and what parts are sedentary.
I've read a lot (the forums, the wiki, the manual) but I'm still confused regarding the strategy of disks in Proxmox; not from a ZFS-sense (I think I understand that fine), but more from a Proxmox perspective. I'd appreciate any input available on disk-strategy for boot, VM running, VM ISO storage, VM/CT shared storage, etc, and what parts are best on SSD/HDD.
My old motherboard can't support enough RAM for ZFS so after a couple of months of research I've bought a second-hand Supermicro server (x10srh-cln4f, E5-2620 v3 [upgradeable as necessary], 32Gb RAM [upgradeable as necessary]). The use-case for my outgoing server was as a NAS and services for my household - Owncloud, Emby (will swap to Jellyfin), torrenting, JIRA, etc. I want to move to Proxmox for ZFS and the freedom granted by a hypervisor (and of course because I think it's really cool). Single node scenario.
The motherboard has a built-in PCI LSI 3008 controller (which I've already flashed to IT mode), and my intention is to pass this through to a FreeNAS VM, along with around 16Gb RAM to share my spinning rust. As I say I've spent a lot of time reading up on this so I'm happy with this part of the system.
In terms of storage space aside from the NAS, I don't think I need too much for my services, as my household data (Linux ISOs and documents) will be on the NAS. Torrent downloads will go straight to the NAS. I plan to experiment with a few minimal VMs and will opt for containers where I can. I thought to experiment with Docker Swarm (I've loved reading the recipes on https://geek-cookbook.funkypenguin.co.nz/ha-docker-swarm/nodes/), and this means at least 60Gb assigned to VMs (perhaps alleviated by shared storage?). I want some shared storage for VMs (my reading leads me to sharing this from the host as NFS), and I want any databases to be on SSDs (though the databases will be small, I think - no mass-storage, just what the services need). I want to consolidate databases by type, so a single InfluxDB database, a single MySQL database, etc, available to all relevant services.
What I'm not sure about is the sizes of the shared storage for VMs and booting etc. For the non-NAS side of things I have 2x 3.5 bays remaining in my chassis, sufficient for up to 4x SSD drives. My preferences, which I'm willing to be swayed on, are that the boot drive is a mirrored pair of SSDs, and the shared storage is a mirrored pair of SSDs also. But I can't figure out the sizes necessary.
I'm looking to spend £100, plus or minus about 20%. I'm looking at SSDs at prices such as £18 for 120Gb, £28 for 240Gb, and £46 for 480Gb. I couldn't, for example, afford 4x 480Gb.
Should I get, for example, 2x 120Gb (mirrored) for the boot drive, and 2x 480Gb (mirrored) for the shared storage? Is 120Gb enough for the boot drive? I noticed in a test install that Proxmos partitioned a disk into 100Gb for installation, and the rest for <I'm not sure>.
Or would it be best to use the 480Gb mirror for boot + VMs, and 120Gb mirror for shared storage? I have a feeling that 120Gb would be enough for shared storage, but I'm unsure if I can host VMs on the boot drive.
Alternatively, would it be better to back-share some rust from the NAS for some sort of VM storage? I know that VMs (and certainly databases) will perform better on SSDs than HDDs, but I'm not sure what parts of proxmox are moving and what parts are sedentary.
I've read a lot (the forums, the wiki, the manual) but I'm still confused regarding the strategy of disks in Proxmox; not from a ZFS-sense (I think I understand that fine), but more from a Proxmox perspective. I'd appreciate any input available on disk-strategy for boot, VM running, VM ISO storage, VM/CT shared storage, etc, and what parts are best on SSD/HDD.
