HW for Proxmox and NAS

[ajocius]

Purpose:

System to securely keep photos, backups, security camera footage (1 camera for now, movement triggered recording, max 30 days), other important data files (2TB x 2 should be sufficient). No film or music libraries. No gaming. And Proxmox server for few VM’s and containers as listed below. Main question: should that be one PC given limited requirements for HDD volume or two separate PC’s.

I have some old pc’s (full size towers) that I can possibly use for NAS purposes, but given the old motherboards and PSU’s I am afraid those will be power hungry. I would think that NAS SW is not RAM hungry, old PC’s I have are with 4-8GB RAM. This however would be rather cheap solution, only need to buy 2 HDD’s.

I do not think my old pc’s are suitable for Proxmox solution that I want. I would target 64-128 GB RAM, power sufficient system. I would use 512GB M2 for system. I can go with full size motherboard, place is not an issue, other factors are more relevant.

Having two separate PC’s (NAS + Proxmox) means two power using systems, requires quick connection between (reckon 1gb LAN card is sufficient, 2,5 gb would be additional investment).

So, should I target 1 or 2 PC’s to meet my needs??? Pros/cons and your thoughts are appreciated. Below are listed intended use.

NAS

RAID for max security – which RAID?

TrueNAS, Openmediavault, Unraid. Seems like TrueNAS is selected option, it requires identical HDD’s, not other cons vs other SW solutions.

2TB*2 or similar.

Minimal (private video), few hours per year. 30 days of security camera footage, triggered by movement.

No film, audio library.

Photo library to hold historical pictures as well as copies from mobile’s.

Backups

Installation files for owned programs and shareware

Proxmox

VM’s : Home Assistant running 24/7. Some test VM’s with Windows, Linux running occasionally for test purposes.

Frigate 24/7, Photoprism, Jellyfin, Unifi controller.

Will be additional containers, but do not expect anything recourse hungry.

 

[Dunuin]

I would think that NAS SW is not RAM hungry, old PC’s I have are with 4-8GB RAM. This however would be rather cheap solution, only need to buy 2 HDD’s.

Depends...TrueNAS for example wants 8+ GB RAM, PVE wants 2+ GB, Frigate needs some GBs of RAM (like 4+ GB). You want some free RAM to not OOM on spikes. If you buy hardware, get at least 32GB of RAM. 16GB RAM may work if you skip the idea of VMs and work with LXCs and use something more simple as a NAS like OMV with mdadm+ext4.
Also keep in mind that PVE is a pure hypervisor. It doesn't come with any NAS functionalities.

So, should I target 1 or 2 PC’s to meet my needs??? Pros/cons and your thoughts are appreciated. Below are listed intended use.

Like you already said, 2 smaller servers consume more electricity than a single bigger one. But what you also should keep in mind is redundancy. In case a system fails, are you fine with not being able to CCTV or access your files for a week or two. Diagnosing the problem, ordering new hardware and setting everything up again might take some time. With two smaller identical PVE nodes and one failing, you could move the most important VMs and hardware to the working node.

RAID for max security – which RAID?

TrueNAS, Openmediavault, Unraid. Seems like TrueNAS is selected option, it requires identical HDD’s, not other cons vs other SW solutions.

Make sure not to buy SMR HDDs. And for ZFS (which TrueNAS uses) it is also recommended to have ECC RAM. Otherwise you can never trust the checksumming to tell you if there is corrupted data or not.

For Frigate and Jellyfin you probably also want 1 or 2 GPUs to passthrough for HW accelerated video encoding/decoding. And a coral TPU for Frigate.

 

[ajocius]

Depends...TrueNAS for example wants 8+ GB RAM, PVE wants 2+ GB, Frigate needs some GBs of RAM (like 4+ GB). You want some free RAM to not OOM on spikes. If you buy hardware, get at least 32GB of RAM. 16GB RAM may work if you skip the idea of VMs and work with LXCs and use something more simple as a NAS like OMV with mdadm+ext4.
Also keep in mind that PVE is a pure hypervisor. It doesn't come with any NAS functionalities.


Like you already said, 2 smaller servers consume more electricity than a single bigger one. But what you also should keep in mind is redundancy. In case a system fails, are you fine with not being able to CCTV or access your files for a week or two. Diagnosing the problem, ordering new hardware and setting everything up again might take some time. With two smaller identical PVE nodes and one failing, you could move the most important VMs and hardware to the working node.


Make sure not to buy SMR HDDs. And for ZFS (which TrueNAS uses) it is also recommended to have ECC RAM. Otherwise you can never trust the checksumming to tell you if there is corrupted data or not.

For Frigate and Jellyfin you probably also want 1 or 2 GPUs to passthrough for HW accelerated video encoding/decoding. And a coral TPU for Frigate.

Appreciate your feedback. I understand that you recommend to keep two separate PC's considering redundancy, makes sense, haven't thought about it. I could then use older motherboard with 8GB RAM for TrueNAS. I just need to find efficient PSU and hopefully low HDD power usage when idle (besides movement triggered CCTV recordings there should be very occoasional use of HDD for backups, photos, etc). Separate pc that I target to have for Proxmox would have min 64GB RAM, possibly 128GB RAM. I do have coral TPU it is intended to be used with Frigate. I will however need to ready more about GPU's requirements for my set up. Video watching is not going to be high, I tested it on my Proxmox server running on oldish laptop, good enought to review CCTV recordings, could be a bit more responsive for video watching. Would you think 1gb ethernet connection is sufficient to connect Proxmox PC to TrueNAS PC for my needs?

 

[Dunuin]

Would you think 1gb ethernet connection is sufficient to connect Proxmox PC to TrueNAS PC for my needs?

If you only want to store cold data on it, yes. But used SFP+ NICs and DAC aren'T that expensive and would allow for 10Gbit.

For redundancy you usually would use 3 servers. 2x PVE nodes (with for example ZFS replication) + 1x PBS that also works as a qdevice.

 

[ajocius]

Hi, to get 10Gbit I would need two ethernet cards and then switch able to handle that speed right? I will need to google around what is ZFS replication and what is PBS, qdevice.

 

[louie1961]

I have a personal preference for stand alone/bare metal devices for NAS purposes. I prefer this so that I don't have to do hardware pass through and the equipment can control the drives natively. I have a Terramaster F2-423 running TrueNAS scale. I find the Terramaster stuff to be good hardware/terrible software. But it is super simple to install TrueNAS, Openmediavault, Unraid, etc. on these machines. It uses 9 watts at idle. Seems like a good fit for your purposes.

My main Proxmox node is a HP Elitedesk 800 G9 with a i5-12500T processor, with 64 gb of memory. I am running 3 instances of Wordpress (each in their own Debian 12 VM), Nextcloud (also in a VM), and 2 other Debian VMs running 14 different docker containers. I have a ton of headroom left on this machine. I also have a second Proxmox node built on a N100 NUC style machine. I run 4 Debian instances on this box for things like OMV, Ansible, etc. Same story with this machine. It is not resource constrained for my uses. All of this is to say that you may not need as big of a Proxmox machine as you think. My HP elite desk uses 12 watts at idle, even with a 10 gbe NIC installed. My N100 box uses 6 watts at idle. If you have an interest in a low power Proxmox build, you should check out this video from Serve The Home

 

[Dunuin]

Hi, to get 10Gbit I would need two ethernet cards and then switch able to handle that speed right? I will need to google around what is ZFS replication and what is PBS, qdevice.

With only 2 or 3 nodes you don't need a 10Gbit switch. You could directly connect two servers via a single port NIC or 3 servers via three dual-port NICs. Got my single port 10bit NICs for 30€ each and the DAC cables for 10€.