Homelab: VM Machine + NAS using Proxmox

[Dom.sudo-engineer]

Hello!

I'm looking to build a low-powered, small footprint, VM machine that also works as my NAS using Proxmox. I'm in dire need of advice.

• NAS Usable storage: 4TB+
• 64GB RAM
• 6c/12t or more CPU
• Budget: $1000 (including drives)
• Small form factor
• Low power consumption
• Durable

My original bad idea:
Pick up a HX90, EliteMini X500, or Intel NUC, put a single 5TB 2.5 inch HDD (or 4TB SSD), pass through the drive to a guest and setting up the storage as a network share. Obviously, wouldn't be great.

Next idea
Pick up a HX90, EliteMini X500, or Intel NUC, connect external storage, pass the external storage to guest and setting up the storage as a network share. Also, not great.

My dream of a Mini Computer + NAS seems DOA. VMs are going to be running a few services, NAS is pure storage - won't be "working" from the NAS at all. I'm now thinking of just building something. I live in a small apartment so I need whatever I build to be small and as power efficient as possible. I also know that using Proxmox to host a NAS isn't ideal either, but I like Proxmox so I'm hoping I can make this work.

 

[nick.kopas]

Have you thought about picking up a used business class workstation from eBay? Maybe something like a Dell Optiplex?

You could run Proxmox off an M.2 drive and passthrough a couple mechanical drives to a VM for your NAS.

 

[Dunuin]

Also think about a backup strategy. If your PVE host doesnt got enough space to install several HDDs you need to buy several USB HDDs and store them somewhere too. Might be better to just build a small MiniITX PC that fits your NAS needs. By the way...HDDs that use shingled magnetic recording (SMR) shouldn't be used in a server. Biggest 2.5" HDD you can get with conventional magnetic recording (CMR) is 2TB. I would atleast get a case where you can put 3.5" HDDs into it.

 

[Dom.sudo-engineer]

Also think about a backup strategy. If your PVE host doesnt got enough space to install several HDDs you need to buy several USB HDDs and store them somewhere too. Might be better to just build a small MiniITX PC that fits your NAS needs. By the way...HDDs that use shingled magnetic recording (SMR) shouldn't be used in a server. Biggest 2.5" HDD you can get with conventional magnetic recording (CMR) is 2TB. I would atleast get a case where you can put two 3.5" HDDs into it.

Awe. Right. Thanks for this info. Didn't even think about the SMR nonsense.

I know next to nothing about MiniITX, will there be a Mobo that has 2 M.2 slots that fits the form factor?
M.2 Proxmox OS
M.2 Proxmox VM
2x Sata 3.5 HD - mirrored

Would give me some redundancy on the NAS which my idea' were missing...

 

[Dunuin]

There are some consumer boards that also would support ECC RAM. For example "GIGABYTE X570 I AORUS Pro WIFI". But it only allows M.2 up to 2280 length (and enterprise SSDs are highly recommended for server workloads and these are usually 22110 in length) and the second M.2 slot is on the back of the boards, so you can't really cool that SSD and they can get quite hot and might thermal throttle when reaching around 80 degree. More options with two M.2 slots you got when choosing a MicroATX formfactor.

 

[Dom.sudo-engineer]

There are some consumer boards that also would support ECC RAM. For example "GIGABYTE X570 I AORUS Pro WIFI". But it only allows M.2 up to 2280 length (and enterprise SSDs are highly recommended for server workloads and these are usually 22110 in length) and the second M.2 slot is on the back of the boards, so you can't really cool that SSD and they can get quite hot and might thermal throttle when reaching around 80 degree. More options with two M.2 slots you got when choosing a MicroATX formfactor.

Sounds like I'm going MicroATX. I'll spec out a build and see what I can come up with.

 

[Dom.sudo-engineer]

Have you thought about picking up a used business class workstation from eBay? Maybe something like a Dell Optiplex?

You could run Proxmox off an M.2 drive and passthrough a couple mechanical drives to a VM for your NAS.

Good advice, the services I'm planning to run on the VMs are critical so I'd rather not put them on used hardware. I'll look into this route and see what I can find.

 

[Dom.sudo-engineer]

Just had another thought. And this question is going to reveal how little I truly know...

If I'm hosting a NAS via a Proxmox guest then it's technically not safe for me to run my Proxmox backups to that NAS because I won't be able to access the NAS if my pve goes down, correct? Or is there a way around this?

 

[nick.kopas]

Just had another thought. And this question is going to reveal how little I truly know...

If I'm hosting a NAS via a Proxmox guest then it's technically not safe for me to run my Proxmox backups to that NAS because I won't be able to access the NAS if my pve goes down, correct? Or is there a way around this?

Depends. If you're passing through a controller or drives... any ZFS pool you build on those drives can be shuffled around pretty easily in a disaster. But if you backup your NAS to a external drive (or cloud storage) using ZFS send, you'll be good either way!

 

[Dom.sudo-engineer]

Depends. If you're passing through a controller or drives... any ZFS pool you build on those drives can be shuffled around pretty easily in a disaster. But if you backup your NAS to a external drive (or cloud storage) using ZFS send, you'll be good either way

Awesome. Okay, maybe this still is an option...

 

[Dunuin]

But you won't be able to backup your NAS VM, as your backup target would be that NAS VM. So you atleast would need to backup that NAS VM somewhere else. And you shouldn't backup VMs to the disk that also stores your VMs. So you need atleast some kind of third storage.

Btw...if your services are critical you shouldn't use consumer hardware. User new consumer hardware isn't better than using used enterprise hardware. And with new enterprise hardware your 1000$ limit is quickly reached. Then it would be more something like this and you end up with 2000$+: https://www.thomas-krenn.com/en/products/tower-systems/microserver/microserver-mi106plus.html

 

[Dom.sudo-engineer]

But you won't be able to backup your NAS VM, as your backup target would be that NAS VM. So you atleast would need to backup that NAS VM somewhere else. And you shouldn't backup VMs to the disk that also stores your VMs. So you need atleast some kind of third storage.

Btw...if your services are critical you shouldn't use consumer hardware. User new consumer hardware isn't better than using used enterprise hardware. And with new enterprise hardware your 1000$ limit is quickly reached.

This isn't safe setup?
2x HDD Mirror for the NAS
1x m.2 ssd for proxmox OS
1x m.2 ssd for VMs
Do my proxmox VM backups to the NAS....but then where do I backup the NAS VM. Awe, I see the issue. Since I'm probably going MicroATX I should have room for another internet 2.5 drive...and maybe at this point I should be getting a HBA for the NAS and pass it through to the NAS VM so I don't have to worry about connecting....5 drives to the motherboard.

Rip low power dreams...

Critical was too strong a term, but noted. I think i meant more, I don't want to deal with higher risk of hardware failures because it's annoying when those services go down. Only one service I run is something I would actually consider critical because it's work related.

 

[Dunuin]

Jep but lets say you want to run TrueNAS in that VM. If you follow the recommended hardware suggestions provided by TrueNAS you need ECC RAM, a couple of (for parity, otherwise your data might silently corrupt over time) CMR HDDs that are 24/7 rated , Enterprise/Datacenter grade SSD with powerloss protection, ...and the hardware recommendation for PVE isn't much different. If you want a reliable homeserver you need reliable hardware. It will run with consumer hardware...but it's not recomended as it is not that reliable and you might risk downtimes or even loosing your data.
You can easily spend more than 1000$ just for a single enterprise SSD. You spoke about a 4TB M.2 SSD. The cheapest enterprise grade 4TB M.2 TLC SSD (Samsung PM9A3) costs 733$.
If you are on a budget you can either buy reliable second hard enterprise hardware (that might not be that reliable anymore because of its age) or you buy new consumer hardware that isn't realiable because of all the missing enterprise features like powerloss protection, ECC and so on that is needed for data integrity and a stable operation. I personally prefer second hand enterprise hardware. If the hardware fails I replace the failed parts. But atleast the data is safe while it is running. Most reliable bang for the buck you get by running multiple second hand servers in a cluster so it doesn't matter that much if a server would fail. But that again isn't very energy nor space efficient.

If you are planning a road trip across africa, better get something well tested and reliable like a old jeep/landrover that was designed to last long, be easily repairable and resist the harshest conditions and not something only designed for the city like a new mini/smart, even if it might consume less fuel and can fit in smaller parking lots.

 

[Forssux]

Jep but lets say you want to run TrueNAS in that VM. If you follow the recommended hardware suggestions provided by TrueNAS you need ECC RAM, a couple of (for parity, otherwise your data might silently corrupt over time) CMR HDDs that are 24/7 rated , Enterprise/Datacenter grade SSD with powerloss protection, ...and the hardware recommendation for PVE isn't much different. If you want a reliable homeserver you need reliable hardware. It will run with consumer hardware...but it's not recomended as it is not that reliable and you might risk downtimes or even loosing your data.
You can easily spend more than 1000$ just for a single enterprise SSD. You spoke about a 4TB M.2 SSD. The cheapest enterprise grade 4TB M.2 TLC SSD (Samsung PM9A3) costs 733$.
If you are on a budget you can either buy reliable second hard enterprise hardware (that might not be that reliable anymore because of its age) or you buy new consumer hardware that isn't realiable because of all the missing enterprise features like powerloss protection, ECC and so on that is needed for data integrity and a stable operation. I personally prefer second hand enterprise hardware. If the hardware fails I replace the failed parts. But atleast the data is safe while it is running. Most reliable bang for the buck you get by running multiple second hand servers in a cluster so it doesn't matter that much if a server would fail. But that again isn't very energy nor space efficient.

If you are planning a road trip across africa, better get something well tested and reliable like a old jeep/landrover that was designed to last long, be easily repairable and resist the harshest conditions and not something only designed for the city like a new mini/smart, even if it might consume less fuel and can fit in smaller parking lots.

ngo's tend to buy Toyota

 

[Forssux]

I moved from QNAP to Truenas core and now I'm on Proxmox.
At home I have Proxmox main 24/24 --> Proxmox backup 1/week --> Hard drive 1/2weeks
Everybody's situation is different and though you'll get plenty and correct advise to do things linke in an enterprise environment.
I would still like to give you my setup. It's a "homelab", nobody but my family is accessing this gear.
I bought a Dell R520 LFF (LFF is for 3.5" disks SFF is for 2.5") my machine had 80G and several hard drives and 4 NIC's the cpu is 2 x a Xeon(R) CPU E5-2470 0 @ 2.30GHz. I also bought a 10G NIC and recently I installed a 100€ 1TB NVMe with two partitions on it. One for the VM's and Containers and the other partition is the backup of the first (for configuration screwups ) all is back-up to the zfs part and to another once a week powerd up machine and eventually bare HD's.
The main server costed me400€ with rackrails included +100€ 10G NIC and 100€ for the 980 NVMe.
My old storage is on a zfs pool on the main Proxmox and on this bare Proxmox I installed samba to share the content with the other pc's in the home. Proxmox became a NAS.
This is frowned upon for good reason. In an enterprise/home where place and money isn't that of a big deal one should separate the storage from
the hypervisor.
Some people put Truenas on a VM on Proxmox which is fine, but I still need to find out the benefices of this approach to the Samba on Proxmox.
The Power consumption is 250W If I may believe iDrac.