Should I logically separate my bulk data (Linux ISOs) onto my HDDs and hot data (VM images, Docker volumes, etc) onto my SSDs?

[kayson]

I currently have all of my HDD storage combined as one mount (44TB), and all the SSD storage combined as another (2TB). I put anything that really needs speed, like VM disks, Docker volumes, nfs-exported home directories, etc on the SSDs, and all ISOs, images, backups, etc on the HDDs. In a sense I'm micro managing the placement of data.

I'm about to move everything into a hyperconverged 4-node proxmox cluster, and I'm wondering if I should keep doing the same. The alternative would be to have a single storage and use the SSDs as cache in front of the HDDs. The upside is I don't have to micromanage (which isn't really that bad), but the potential downside is relying on the cache implementation, whether that's ceph, LVM, zfs, etc. Not sure how well it will work.

Any thoughts?

 

[Johannes S]

Well this depends heavily on your usecase and which comprimises you are willing to take in terms of performance/costs etc.

I personally prefer to seperate between VM data (operating systems and applciation (including configuration e.G. for Jellyfin/plex,paperless), bulk data ( the data managed by the applications eg. jellyfin/plex/paperless media) and disposable (because I just need to setup things, saving it locally is just for convenience in case of disaster I can always redownload it e.g. iso images).
The main reason for seperation are the available backup options for me in my homalb: My bulk data (stuff on my TrueNAS instance and notebook) has quite grown over the years so a cheap cloud storage provider (like my Hetzner storageboc) is the way to go. I back it up with restic so PBS never ever touch it.
Thus any VM working with it gets at least two virtual storage discs: One for the VM or LXC os (Debian in my case) and applications (including their configurations) and another one for the actual data. So I can use PBS for backing up the VM and LXC OS/application setup and restic for the actual data.

This works fine for me but for you the story can be quite different: For example if you actually doesn't have much of data anyhow or it can be regenereated any time (e.g. your mp3ed your audio cd collection but you still have your audio cds so you coud repeat it any time) then it might be actually less of a effort and much easier to just keep everything in one storage and backup it directly to PBS.
On the other hand seperating things quite from the stuff make it easier to seperate them later on the backup too.