Small Server Setup

[newfang]

Hi,

I'm currently setting-up small server for my univ. department internal purposes. I was hoping I can get some advice from more experienced users.

Hardware is:
Intel Xeon - 2,2 GHz, 10 Cores, 20HT
16 GB RAM ECC
Asus server motherboard with no hardware RAID
3x1TB HDD, 1x2TB HDD

I will be fighting for purchase of extra 1TB or 2TB and some (probably cheap) HW RAID controller, but budget is tight and for now on - I have to deal with mentioned specs. I've decided for proxmox because of need for both linux and windows server services, also plans to add some server units in future (probably distant).

Estimated max users: 15-20, about 5-10 at once
Estimated containers:
Nextcloud - approx 2-3TB,
Gitlab - approx. 200-300GB
Fileserver - approx. 100GB
Webserver - 1 site, approx. 10GB
Estimated vm (on thinpool):
Windows 10 - 100-300 GB (if memory possible, then 2 vm)

Variant 1:
Proxmox at 2x16GB USB stick, snapshot and dd to 2nd USB stick every week
MDADM RAID5 (3x1TB HDD), LVM 2TB
RAID for linux containers (500GB) and windows vm (300GB), snapshots (200GB), remaining (1TB), nextcloud at 2TB LVM.
This variant cons: I'm afraid about USB stick wear, pros: clear disk space division.
I'm currently testing variant 1 at idle and takes approx. 10GB of RAM, io delay 0-5%.

Variant 2:
Proxmox at RAID1 with dual boot if one disk fails
MDADM RAID1 (2x1TB HDD), LVM (1TB + 2TB)
RAID for proxmox and containers (500GB), windows vm (300GB), snapshots (200GB), nextcloud at 3TB LVM.
This variant cons: more complicated space division, pros: more secure proxmox config.

Which one of those variants you think are better? Are there better options (for. ex. ZFS instead of RAID)? Will machine have enough resources for all of this? Suggestions for future (distant) developments (RAID10 at 4x1TB and RAID1 at 2x2TB)?

Any other ideas also welcome! Thanks!

 

[Rhinox]

My own opition:

1. hw is good, but if you can, get more RAM. 16GB is too little even for all those VMs/containers.
2. forget hw-raid/mdadm/lvm-raid, use zfs-raid (did I say more RAM?).
3. Do not install proxmox on usb/cf/sd (variant 1), not even 2x in raid1.

Proxmox is not like ESXi (loads image from usb, and runs from memory). Proxmox writes to disk quite frequently. I used expensive SLC-based usb, and it died within one year. Use 2x small SSD, but some people here had problems even with consumer-SSD...

 

[newfang]

Hi

Thank you for fast reply. I will try to get some more RAM. I've also moved proxmox to ZFS RAIDZ-1 3x1TB HDD disks.

I have one more question concerning ZFS. First I was thinking about mirrored ZFS 2x1TB with possible proxmox boot from both devices. However, after some tests at virtual machine - It seems that RAIDZ-1 is able to boot - even after removal of a rootfs. Software RAID-5 failed after removal of device with mdadm. How is it possible?

BR