Yet another noob install question...

[MikeWithTheBike]

Hi guys,

I'm about to try Proxmox for the first time, and I have some very basic questions...

I think I've pieced together the info from other posts, but a lot of (good) advice seems to rely on NVMe drives, of which I have zero.

What I do have is a cheap used Dell to play around with, and 5 drives - 2 very cheap SSDs and 3 reasonable 4 TB HDD's.

I'm thinking of using one cheap SSD to install Proxmox and store ISO's, a second identical SSD for VM's (one of which will be TrueNAS) and the 3 HDD's for data.

I understand that a cheap drive for VM's is a bad idea long term, and I'll probably replace that shortly. Assuming I can get everything up and running and I like the setup, I'd wipe the VM drive, use it to mirror the Proxmox drive, and get 2 used enterprise SSD's for the VM's (or NVMe drives on a PCIe card?).

Is that reasonable? Will I kill the cheap SSD's mirroring Proxmox the way I would with VM's? Am I totally missing something obvious? Should I get a separate PCIe controller for the HDD's before I do this (since I understand that I can't pass individual drives through to TrueNAS), or can I set it up this way and then move them to a controller later if I keep this setup?

Thank you for answering all these basic questions that are probably obvious to most of you...

 

[UdoB]

2 very cheap SSDs

With this you should avoid to use ZFS. Go for mdraid instead.

But... I am a ZFS fanboy, so my actual recommendation is to get two used "Enterprise Class" SSDs and install PVE on a ZFS mirror. The "why" is here: https://forum.proxmox.com/threads/f...y-a-few-disks-should-i-use-zfs-at-all.160037/

Then there are two options:

1) two very small SSDs --> only used for PVE, the operation system

2) buy larger ones (1TB or 2TB) to be used during installation for the OS and also for the storage of the VMs. But not for "bulk"-data.

While the recommendation is still to separate "the OS" from VM-storage from User-Data this may be acceptable. Especially on Mini-PCs with a limited number of SATA connectors. (I do this in my Homelab while in my dayjob I do really avoid this.)

 

[MikeWithTheBike]

Thanks! I was assuming it was the VM's that would kill the cheap drives the fastest, but it's PVE - not what I was expecting.

The plan now is to have 2 enterprise drives for PVE (like a pair of 160 GB S3500's, unless someone tells me otherwise) and then a PCIe to NVMe adapter for a pair of NVMe drives running the VM's... Assuming I boot this up on these cheap drives, run it for a week, and like it!

It sounds like the mirrored drives for PVE need to be able to last through a lot or writes, but don't have to be particulalry fast, and the VM's should be running on something faster and that I can pass thorugh to TrueNAS...

For reference, the system supposidly has 8 drive bays... of which is seems 6 work (one is missing the tray, and the other tray is *very* wedged in there...)

 

[Johannes S]

Thanks! I was assuming it was the VM's that would kill the cheap drives the fastest, but it's PVE - not what I was expecting.

The plan now is to have 2 enterprise drives for PVE (like a pair of 160 GB S3500's, unless someone tells me otherwise) and then a PCIe to NVMe adapter for a pair of NVMe drives running the VM's... Assuming I boot this up on these cheap drives, run it for a week, and like it!

It sounds like the mirrored drives for PVE need to be able to last through a lot or writes, but don't have to be particulalry fast, and the VM's should be running on something faster and that I can pass thorugh to TrueNAS...

Exactly, for the OS even HDDs would be ok ( not for VMs of course). Concerning the NVMEs: Please keep in mind that any disk will be gone at some point so Plan accordingly ans be prepared

For reference, the system supposidly has 8 drive bays... of which is seems 6 work (one is missing the tray, and the other tray is *very* wedged in there...)

Than I would go for teo mirros on enterprise-ssds: One for the OS, oben for vms/lxcs, maybe two hdds as a mirror for bulk data. Of course Budget might make this difficult.

 

[MikeWithTheBike]

Much appreciated!

I may have focused too much on the "cheap" criteria... This whole project came about when I was given an old Dell R430 with 128 GB of ram that I didn't really need... But wanted to find a use for! I would love to have Proxmox running a little collection of projects that would be useful (but not essential) at the office - I just worry I won't have the time / skill to manage it! If it doesn't just sit there half done as an "almost" project (I'm sure we all have a few of those), then there aren't any significant budget restraints. Anything that saves me time / frustration is worth the cost!

I'm looking for the most reasonable way to set it up with these cheap drives that happen to be sitting here, and then migrate it over to a more reliable / faster setup assuming it actually gets a little use.

For data I was going to run 3 HDD's in a Raid 5 equivalent (I'm still old and not up to date on ZFS yet... Just wanted 3 drives with 2 drives worth of storage), but it probably makes sense to just run a mirrored pair for now (they're 4 TB drives and I don't need more than 4 TB for the application yet). I already have 3 decent 4 TB HDD's that I was going to use in another project. I was NOT going to have anything more than some test data on this until I confirmed it was going to be a thing and I was using better drives for the PVE / VM's.

If I start with the cheap SSD for PVE and the other for the VM's, can I then mirror those on better drives, pull the cheap drives, and mirror those back to matching better drives? Or am I creating a make-work project and I should just see if I like it on the cheap drives and then do a new install on better drives?

Basically, how much should I do on these cheap drives before I commit and just get the right hardware??