New To Proxmox, Looking for advice (physical drive configuration, hardware utilization, etc.)

808mp5

New Member
Dec 13, 2023
3
0
1
Hello all,
This is my first post and will be the first time I'll be diving (more like crashing) into the Proxmox rabbit hole. I've little knowledge of Proxmox (other than knowing it's a Hypervisor O/S, and ingesting YouTube videos), and little knowledge of Ubuntu/Linux in general. Even with little to no experience I am in no means inept to learning new things. I'm here to learn while crashing and burning. It's nice to meet you all, please take care of me, and thanks in advance.

My current setup is all over the place and i'm looking to consolidate all if not most of it into Proxmox (I'm not looking to repurpose any of the NUCs for example, utilizing them for any type of cluster or HA).
2x Intel NUCs (one windows 11, one Unbuntu LTS) both hosting PMS
1x Intel NUC running Hyper-V core (pfsense NAT for VPN, Sonarr, SABNZBd, Deluged, Radarr, Lidarr, Pi-hole)
4xSynology 8bay NAS each maxed out with 5tb, 14tb, 16tb, and 18tb drives respectively (hosting a Pi-hole and HomeAssistant VM)

My current network setup:
All Ubiquiti. UDM Pro SE, 24 port 10gbe switch, 24 port 2.5gbe switch, Aggregation switch, etc. I think you guys got the idea.

Main use case:
Media Server (Plex Media Server want to run two instances for shared library seperation, *'r apps, tdarr, etc.)
Applications can be VMs or containers. I really don't know how to use containers. The only experience I have with these is with Synology and ditched the idea because it wasn't any fun trying to figure out how to segregate the Docker bridge network so i could pass it thru a vlan on the main network. I do have experience using Hyper-V Core so i'm not that ignorant when it comes to Virtual Machines.

Purposed Hardware: (more money than sense)
CPU: Intel 14900K
Motherboard: Asus W680 Pro-ACE (no IPMI, as I tried the IPMI version and got a dud board the first time)
Memory: 128gb ECC Memory (currently on 6th pass of Memtest86+)
Network: 2x10gbe NIC, 2x2.5gbe NIC, (plus the onboard 2x2.5gbe)
Graphics: Intel Arc A750 (want to add an additional card but deciding what single slot is good)
Drives: 4x Samsung 870 EVO 1TB, 2x Intel D3-S4620 480GB, 2x Samsung 870 EVO 250GB, 2x Samsung 990 Pro 2TB, 1x Samsung Pro 4TB

Misc Additional Hardware: (plan to virtualize some sort of NAS O/S, but haven't decided which one yet)
LSI 9211-8i P20 HBA
Seagate Exos x22 20TB (have 3 so far)

A little more hardware:
I already have 4xSynology NAS that host all the data. So no major storage needed on the Proxmox machine itself besides trying to virtualize a NAS on it. Plan is to maybe install PBS and attach network storage to Proxmox for backup.

***So with the above hardware and information. How would the Proxmox guru's approach this. For example, how would you configure the drives (ZFS, Raid, caching (if even required), segregating the logs, etc). Where would you put the VM storage, where would you install the Proxmox O/S.

I know some of the information above is redundant but my brain is idling at 75mph at the moment. Any and all advice is greatly appreciated.

****The drive configuration is what's got me thinking the most. But this is what i'm thinking.
Proxmox O/S boot: 2x Samsung 870 EVO 250gb ZFS Mirror
Main VMs PMS: 2x Samsung 990 Pro 2TB ZFS Mirror
Additional VMs: 4x Samsung 970 EVO 1TB ZFS Raid10
ZIL/SLOG Offload: 2x Intel D3-S4620 480GB ZFS Mirror
Not sure yet: 1x Samsung 990 Pro 4TB
Yay/Nay or am I just lost in the sauce.
 
Last edited:
First you would need to decide if you want a cluster or not. With 3 NUCs you probably want a cluster. Then with a cluster there would be the question if you want shared storage (Ceph or NFS) for your VMs or ZFS with replication.

By the way...only those S4620 are enterprise SSDs and recommended for ZFS/ceph or in general for any server workloads.
 
  • Like
Reactions: 808mp5
First you would need to decide if you want a cluster or not. With 3 NUCs you probably want a cluster. Then with a cluster there would be the question if you want shared storage (Ceph or NFS) for your VMs or ZFS with replication.

By the way...only those S4620 are enterprise SSDs and recommended for ZFS/ceph or in general for any server workloads.
Thank you for the response. As of right now i don't plan to run a cluster, HA, or utilize the NUCs after setting up the Proxmox O/S. I was thinking to use ZFS for all the pairs of drives. From my understanding the CoW features of ZFS is what kills drives and my plan was to dedicate the Intel drives for the ZIL/SLOG. In all honestly the drive configuration is what's holding me back. I'm not sure if i should use ZFS or not with all the talk of write amplification. Also even if dedicating the Intel drives for the log, i feel that will hinder the performance of the rest of the drives especially the NVMEs. In the end I really don't know and probably won't know until i try it.

I'll update the first post with an idea of how I plan to utilize the drives.
 
Last edited:
just did a fresh install but not sure what this error is in red. It happens when i run update-grub

Copying and configuring kernels on /dev/disk/by-uuid/F8A2-072E
Copying kernel and creating boot-entry for 6.5.11-4-pve
Copying and configuring kernels on /dev/disk/by-uuid/F8A2-70D6
Copying kernel and creating boot-entry for 6.5.11-4-pve
Found linux image: /boot/vmlinuz-6.5.11-4-pve
Found initrd image: /boot/initrd.img-6.5.11-4-pve
/usr/sbin/grub-probe: error: unknown filesystem.
/usr/sbin/grub-probe: error: unknown filesystem.

Adding boot menu entry for UEFI Firmware Settings
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!