Hard Drive Best Practices

[RuckusVol]

I'm new to Proxmox, homelab, advanced networking, etc. Got a server to replace the mini PC I used to run home assistant and a bunch of add-ons. My primary goal is to learn and have fun doing a lot of new things. Secondary is having a much better setup than the mini PC that crashes under load from the amount of stuff I have it do - NVR, media server, lots of home automation, etc.

I've read and watched a lot of videos. I'm still unclear on best practice for drive allocation, both hardware and software.

My best understanding now would be something like this:
OS: 2x smalls SSDs (like 128GB) ZFS Raid 1 (Z1?)
VMs: mid size SSD 1-2TB. Unclear if 2 in RAID or single with regular backups is preferred.
Storage: whatever size needed in ZFS RAID of some form.

I probably did something not ideal, but I got 2 480GB S4510 drives. That seems to big for the OS and maybe too small for VMs unless 1TB from RAID 0 or something similar is a smart idea. I'm assuming combining them and running OS and VMs off them is a bad idea.

So my question is what's the right way to do it, and is there a way to do that with the 480GB drives I got that isn't wasting most of them?

I'm brand new and don't get offended. So feel free to be as direct as you wish. I appreciate any input I can get.

 

[jdancer]

Considered best practice to mirror the OS using RAID-1. In my case, I use two small drives to ZFS RAID-1 Proxmox itself.

As for the rest of the storage, it depends on the use case.

If a single drive, I do use ZFS RAID-0 (stripe), so can use snapshots, rollbacks, compression, error checking. etc. Obviously, back the data/VMs.

If 2 drives, ZFS RAID-1. If 4 drives, ZFS RAID-10. If 6 drives, ZFS RAID-50. if 8 drives, ZFS RAID-6 (RAIDZ2).

If you want the best IOPS, then ZFS RAID-10. Requires minimum 4 drives and up.

 

[RuckusVol]

Considered best practice to mirror the OS using RAID-1. In my case, I use two small drives to ZFS RAID-1 Proxmox itself.

As for the rest of the storage, it depends on the use case.

If a single drive, I do use ZFS RAID-0 (stripe), so can use snapshots, rollbacks, compression, error checking. etc. Obviously, back the data/VMs.

If 2 drives, ZFS RAID-1. If 4 drives, ZFS RAID-10. If 6 drives, ZFS RAID-50. if 8 drives, ZFS RAID-60 (RAIDZ2).

If you want the best IOPS, then ZFS RAID-10. Requires minimum 4 drives and up.

This is very helpful. Thank you.

So maybe my best path would be:
OS - 2 small drives in ZFS RAID-1
LVM - 2 480GB drives in ZFS RAID-0 (with regular full backups)

How big for the OS drives is big enough? Is 128GB more than will ever get used? 256 better?
Would 480GB maybe work for VMs in ZFS RAID-1? Or will that get blown through really fast? I've never run a VM machine so I have no idea.

 

[jdancer]

This is very helpful. Thank you.

So maybe my best path would be:
OS - 2 small drives in ZFS RAID-1
LVM - 2 480GB drives in ZFS RAID-0 (with regular full backups)

How big for the OS drives is big enough? Is 128GB more than will ever get used? 256 better?
Would 480GB maybe work for VMs in ZFS RAID-1? Or will that get blown through really fast? I've never run a VM machine so I have no idea.

I believe 64GBs is the minimum. Don't think 32GB is enough.

I do have a server using 100GB SSDs to ZFS RAID-1 Proxmox.

So, 128GB is plenty.

Since ZFS is a volume manager just like LVM, I would skip the LVM and use ZFS. You can JDOD ZFS, ie, ZFS RAID-0. Of course, backup it up.

 

[maartenv]

My setup is similar, but I'm also a novice in Proxmox. I have 3 nvme ssd, 2 sata ssd and 2 HDD, crammed in a nice optiplex sff.

I also installed proxmox on the 2 sata ssd as ZFS Raid 1.
The vm's will be running on 2 nvme ssd ZFS Raid 1.
The 2 HDD's will be for slower storage ZFS Raid 1.
I want to use the last nvme ssd to use as cache for the HDD but I'm still puzzling/googling how to do that.

However I'm not sure this is the way. Please advice.

I'd however recommend against raid 0 with backups. It is a lot of work to get things back online and you usually still have data loss.



P.s. I used to have proxmox with just a single ssd for installation and vm's and a single hdd for slow storate, but the SSD slowly died at one point and caused a lot of corruption. It took quite some work to bring it back online even tough i had backups. That's why im going for more redundancy.