Initial Setup of Proxmox Homelab (Services and Configuration)

[tja]

Hey folks,

I'm looking to set up Proxmox on bare metal and create multiple virtual machines (VMs) for different services like Ubuntu/Debian servers for various purposes. My aim is to use this as a general-purpose homelab, running services like firewall, VPN, media server, file hosting, networking/monitoring and some FOSS software. I also want to access some services in my private network from the outside (maybe via Cloudflare Zero Trust Tunnels).

Here's the current hardware I'm working with: HP EliteDesk 800 G5 SFF (specs: https://support.hp.com/us-en/document/c06414352)

- CPU: Intel i5 9500
- GPU: Intel UHD Graphics 630
- PSU: 250 W (80 PLUS Platinum)
- RAM: Crucial Pro DDR4 2666 (2 x 32GB)
- SSD SATA: Crucial MX500 (1 x 500GB)
- SSD NVME: Crucial P3 (2 x 2TB)
- HDD: Seagate Exos X18 (2 x 16TB)

For services, I'm planning to use:

- Firewall: OPNsense
- VPN: OpenVPN / WireGuard - what is the more robust solution?
- Media: Jellyfin, Sonarr, Radarr, Prowlarr
- File Hosting and Synchronization: Nextcloud, PhotoPrism
- Networking and Monitoring: AdGuard / PiHole, Cloudfare, Nginx Proxy Manager / Traefik, Uptime Kama, Grafana
- Password Manager: Vaultwarden
- Dashboard: Heimdall / Homarr
- Software Development: GitLab
- Download Clients: Deluge
- Misc: LocalAI, Paperless, Kasm Workspaces, Kavita, Audiobookshelf, FreshRSS, Invidious (YouTube), Teddit (Reddit)

I'm comfortable with Docker, so I'm thinking of running multiple Docker hosts in dedicated VMs (like Ubuntu/Debian server) and managing the services with Portainer.
However, I'm unsure about using Kubernetes and Rancher instead. Is it worth learning these tools for enabling HA clusters which would result in a more robust setup?

I'm also undecided on storage / backup solutions. I've heard good things about TrueNAS Scale since it has built-in periodic snapshoting and simplified file sharing system (NFS) but using it as a guest VM might involve extra effort using an HBA PCIe card to passthrough the disks (HDDs). Is it worth it?

As for file systems I'm also not an expert. What would be a reasonable configuration for this system?
I've heard ZFS has great features but consumer-grade SSDs might not be ideal (wearing on heavy workload), so I should only use ZFS for the HDDs. Any thoughts on this?

In terms of VM setup, should I dedicate a VM for each service category for better isolation and file sharing? Are there any best practices you recommend?

Lastly, do you think the CPU and RAM I have are sufficient for these tasks?
What are your thoughts on the assigned storage dedication (SATA SSD (EXT4) for boot drive, NVME SSDs (EXT4) for VMs, HDDs (ZFS) for file storage and backups)?

Thanks a bunch for your input!

 

[Dunuin]

VPN: OpenVPN / WireGuard - what is the more robust solution?

OpenVPN is the very old well supported standard. Wireguard is modern, way faster and easier to set up.

- File Hosting and Synchronization: Nextcloud, PhotoPrism

I don't get why people like PhotoPrism. Didn't found a good selfhosted Photo Management Software yet. DigiKam got all the nice features but isn't selfhosted (except you set up a VM for an external MariaDB and NAS) and Piwigo looks best so far for a seflhosted photo management but heavily depends on community plugins for usefulness which is always problematic when updating. And even with plugins that add stuff like geotagging I'm missing features like being able to quickly scroll through picutres in fullscreen, to playback videos or to find duplicate pictures based on fingerprints instead of only on hashes/date/size. But PhotoPrism was terrbile. I think it is only hyped because of that AI stuff. But the AI tagging of photos and face recognition wasn't working well. Wouldn`t be a problem if they would allow you to tag/organize your photos yourself. But they didn't even got the most basic stuff like bulk metadata editing. A pain if you import 100.000 photos that are already well structured and you have to tag them one by one.
Nextcloud is great, but I would highly recommend to set that up yourself from scratch. There is so much stuff you should do to harden security that all those turnkey solutions and docker containers are missing.

I'm also undecided on storage / backup solutions. I've heard good things about TrueNAS Scale since it has built-in periodic snapshoting and simplified file sharing system (NFS) but using it as a guest VM might involve extra effort using an HBA PCIe card to passthrough the disks (HDDs). Is it worth it?

I still prefer TrueNAS Core. And I always got problems with updates breaking important function that then only get fixed after months. But if you need more advanced features it's in my opinion still the best choice. I especially like the SMB with shadow copies, all the data integrity of ZFS and the replication. Always buy your hardware and set up your NAS twice, so you can replicate all data from one to the other NAS for a recent first local backup.

I've heard ZFS has great features but consumer-grade SSDs might not be ideal (wearing on heavy workload), so I should only use ZFS for the HDDs. Any thoughts on this?

If you care about your data then definitely ZFS. But yes, those SSDs won't be great and I would replace them with something proper with a power-loss protection and 1+ DWPD.

In terms of VM setup, should I dedicate a VM for each service category for better isolation and file sharing?

Best you got a VM for each single service. Makes it more secure, much easier to backup, migrate, throw away and keep your services updated. But not always possible. Especially when PCI passthrough is needed like one service needing a GPU for transcoding and another service for AI acceleration.

Lastly, do you think the CPU and RAM I have are sufficient for these tasks?
What are your thoughts on the assigned storage dedication (SATA SSD (EXT4) for boot drive, NVME SSDs (EXT4) for VMs, HDDs (ZFS) for file storage and backups)?

RAM is more on the low side especially if you want to run TrueNAS. Would give that TrueNAS VM at least 16GB RAM if you care about performance.
I wouldn't use a single boot disk. Keep in mind that it is still not possible to backup a PVE host nor to easily export/import its configs. I personally wouldn't use anything that isn't at least mirrored. Would be better to share the NVMe SSDs for boot + VM storage in case it is not an option to get another SATA SSD for mirroring.