Homelab Build Questions

[5mart3ch]

Hi, rebuilding homelab. Before I commit, looking for suggestions. Planned hardware setup looks like this:

System 1 - Main PVE:

• Supermicro X10, Xeon E3 CPU, 32GB ECC RAM
• 600GB Intel 320 SSD
• 2x10GbE, 4xGbE

System 2 - NAS PVE

• Lenovo TS440, Xeon E3 CPU, 32GB ECC RAM
• 600GB Intel 320 SSD
• 10X 4TB HDD (ZFS RAIDZ2 on HBA 8-ports, mainboard 5-ports)
• 2x10GbE, 1xGbE

System 3 - Low power PVE (for quorum):

• Raspberry Pi 3

This will be used for homelab, web development, video surveillance, plex media server, FreePBX/asterisk, Home Assistant, etc. The goal is to have reliable network and system services and fast redundant storage. A pfSense VM will handle networking (firewall, OpenVPN, VLAN, etc.) and Windows Server VM for Active Directory (with DHCP and DNS).


Questions:

1. Will powering down or restarting a host (e.g., PVE-1) bring down networking services (i.e., no DHCP, DNS, etc.)?
2. Would using a Raspberry Pi limit performance or functionality in any way? The main reason for use of the RPi is the low 6W power consumption.
3. Is clustering with HA/live migration possible?
4. Is it recommended to have PVE on HDD and VMs on SSD? Is it to allow for a fast recovery of host system in case failure or corruption? Is HDD for PVE sufficient or also on another SSD?
5. Where to put VMs for fastest performance? I presume will be slower on the NAS and to avoid it.
6. When to use Ceph or GlusterFS?
7. Do I need SSDs for L2ARC or ZIL? Not doing any high transaction database.
8. Better to have RAIDZ2 with 6X HDDs and RAIDZ2 with 6X SSD. Put all storage on NAS?
9. Other ideas?

 

[wolfgang]

Will powering down or restarting a host (e.g., PVE-1) bring down networking services (i.e., no DHCP, DNS, etc.)?

It dependence on you setup if the pfsense is on this node and you can not migrate it, than yes you will have a service interruption.

Would using a Raspberry Pi limit performance or functionality in any way? The main reason for use of the RPi is the low 6W power consumption.

No

Is clustering with HA/live migration possible?

Yes

Is it recommended to have PVE on HDD and VMs on SSD? Is it to allow for a fast recovery of host system in case failure or corruption? Is HDD for PVE sufficient or also on another SSD?

PVE need no fast disk and no service governed the guests.

Where to put VMs for fastest performance? I presume will be slower on the NAS and to avoid it.

Sure SSD are better VM speed.

When to use Ceph or GlusterFS?

In you testup this is not an option. Ceph need miimum 3 nodes with each 3 fast ssd.
GlusterFs need 2 symmetric local storages.

Do I need SSDs for L2ARC or ZIL? Not doing any high transaction database.

Zil makes small writes much faster. L2ARC is often discuss in the forum.

Better to have RAIDZ2 with 6X HDDs and RAIDZ2 with 6X SSD. Put all storage on NAS?

SSD are faster than HDD this also true with ZFS.