Storage Setup Confirmation for 5 drives

P

peaksteve

New Member
May 3, 2025
3
0
1
After learning some things the hard way (mainly consumer SSD's and ZFS) I've been browsing here and other places and think I have a good plan for re-doing my setup. Hoping to get confirmation prior to starting over.

What I'm planning to run:
LXC's
Plex
Torrent
Wireguard
Immich
Arr's

VM
Home Assistant
Opnsense (not 100% but will play around once everything is working smooth)

System - Acer VM4640G
i5-6400 Processor
32 GB of Dual-channel DDR4 2133 MHz
Intel x710 dual nic in PCIe x16 slot

Drives Available:
2x Western Digital 1TB WD Blue 3D NAND Internal PC SSD
2x HP MTFDDAK1T9TBY 1.92TB SATA 6Gb/s
2x WD Red 14TB 7200 rpm HDD

I only have 5 sata ports available so here is what I'm thinking.
1x WD 1TB SSD - Proxmox boot/OS - XFS
2x HP SSD - VMs/CTs - ZFS mirror
2x WD Red - lvm thin mirror - storage for immich/torrents/plex library

Thanks all! I appreciate all of the info here
 
I have 2x additional PCIe 3.0 x1 slots. I did think about adding a SATA controller so I could mirror the proxmox drives. However, in the event the proxmox SSD dies, I can just swap in the spare drive and reinstall proxmox. I'll also be doing nightly backups to my NAS to restore everything from.

The other alternative is to only use 4 drives and put the proxmox OS and VMs/CTs all on the HP SSDs on ZFS with the HDDs the same as mentioned above. This seems like the simpler route but I saw quite a bit of info suggesting against this setup.
 
peaksteve said:
I did think about adding a SATA controller so I could mirror the proxmox drives. However, in the event the proxmox SSD dies, I can just swap in the spare drive and reinstall proxmox. I'll also be doing nightly backups to my NAS to restore everything from.
Click to expand...
If there is only PVE on the disk, you may only need an additional backup of relevant files in /etc and /etc/pve, reinstall and copy back the files. Your VMs can just start again. Reinstalling without the /etc/pve files and you will have to do EVERYTHING again, because all configs are gone.

It's also often unecessary to have a seperate drive for PVE itself. The only upside for me is that you can reinstall without loosing all VM data.
You also mentioned this and that would be the better route. Internal redundancy and less space waste.
peaksteve said:
This seems like the simpler route but I saw quite a bit of info suggesting against this setup.
Click to expand...
As mentioned reinstalling is not possible (or hard for non Linux-natives), but other than that (unless consumer SSDs of course)?

My goto setup is 2x SSDs and 2x HDDs in such a case and ONE pool (dual mirror). The SSDs are the special devices for the pool and by having datasets with special_small_blocks high at least at recordsize level (@Johannes S now I always think of you!), you will write data explicitly to SSDs, otherwise they'll go to the HDD.
 
  • Like
Reactions: UdoB
LnxBil said:
My goto setup is 2x SSDs and 2x HDDs in such a case and ONE pool (dual mirror). The SSDs are the special devices for the pool and by having datasets with special_small_blocks high at least at recordsize level (@Johannes S now I always think of you!), you will write data explicitly to SSDs, otherwise they'll go to the HDD.
Click to expand...
This sounds super interesting... starting my research now!

Link to Proxmox wiki for future reference
 
You must log in or register to reply here.
Top Bottom