What hard-disk configuration

[TIENDER]

Hi,

I want to build a second node for HA. The third voter is a Q-device)
I also want to replace my synology and build a NAS vm in proxmox.

Specs of the server thats running alone now.

i3-8100
P11C-I
32GB ram (Non-ecc)
2x crucial mx500 500GB.

Vm's and dockers running

Network(Pihole and unifi)
Domotica(openhab)
VPN(Openvpn)
Test(For testing)
Windows

The synology (DS718+) has 2 seagate ironwolf 4TB HDD
The synology also has 3 camera's

So for those purposes i'm going to use truenas and zoneminder.

But whats the best way to configure those two nodes.
Is it oke to put one HDD for storage, one HDD for video-footage and one SSD for VM's?
Or should i go for two of those in each node? Cause thats a lot of HDD's en SSD's

Or should i go with 2 SSD in every node, and a singe HDD for NAS and storage?

Also thinking of putting a Xeon 2136 in the second server instead of the i3 8100.

 

[Dunuin]

I dont see how that should work with HA. If you want HA you need a storage that can be used as a shared storage. So there are 3 options:
1.) Ceph: But here you want atleast 10Gbit NICs and a additional NIC only for the sync. And if I'm right you need atleast 3 hosts + qdevice for a HA.
2.) ZFS with replication: This isn't a real shared storage. You need identical pools (and so drives too, so you need to buy everthing twice) and replication will keep them in sync. But if one server goes down you will loose up to 5 minutes of data because the pools won't be perfectly synced.
3.) NFS network share: If both hosts can assess a NAS (so you need a third server) you can store all VM on that NFS share so both hosts access the sane data. But this wont be fast because every IO will need to go through your slow NIC. And this NAS can't be virtualized on one of the hosts. Because if the host with the NAS would go down the other host wouldn't be able to access the share with the VM/LXC anymore.

And I wouldn't use TrueNAS if you just want one NAS drive. TrueNAS is only using ZFS and that wouldn't make much sense if you are only using one drive. There you want atleat 2 drives for a mirror or 3 for a raidz1. The more drives the better. Its also not a good idea to run ZFS ontop of ZFS/ceph or any shared storage because that overhead will multiply.
And ZFS needs alot of RAM. Lets say you got two 4TB drives as a zfs mirror for your host. Now your host needs 10GB RAM for ZFS. And if you want to use TrueNAS using virtual disks stored on that ZFS You again need 6 GB RAM for that ZFS inside your VM. So you got 16GB RAM used just for the storage. And your host and TrueNAS need some RAM too. So another 6 GB abd you are at 22GB just for your host and TrueNAS VM.

For me HA doesn't make much sense unless you use Ceph and for that you want atleast 2 stackable switches, 3 servers with each 2x Gbit + 2x 10Gbit NICs and each 2 small drives as mirror for OS + 3 enterpise SSDs as OSD for Cepf as VM storage. And this is without any storage for your virtual NAS. If you want your virtual NAS to be HA too that again would mean atleat 3 HDDs each. So in total 9 HDDs + 9 enterprise SSDs + 6 consumer SSDs.
Because if you use NFS you again get a single point of failure. Then any of the two PVE servers may fail and HA will work but if your single NAS fails both PVE server will fail.