Small office server tips

[WhiteTiger]

I need to set up a server for an office of only 3 people.
The needs are quite modest, I would activate VMs with Linux servers for Web applications.
A VM is for the pfSense firewall to manage a DMZ.
The other VMs for my IT lab only.
At most I install a Windows 10 VM with games, but nothing particularly complicated and to be used in free time.

I'm on a tight budget, so I was aiming for a Ryzen CPU.
If I can, I buy or assemble a MINI-ITX motherboard, otherwise I mount everything in a tower so at least I have no problems finding space for disks or RAM banks.
For the NICs I was aiming for an expansion card with 4 ethernet ports and possibly a motherboard with 2 ports.

Could you recommend a CPU and a motherboard?

 

[bobmc]

I've setup a couple of small proxmox hosts using AMD processors and always found they worked well providing you don't get too ambitious. For me, having enough RAM and high speed storage is more important than core count and frequency but it will depend on your workload.

I'd spend a bit extra on the motherboard to ensure you have enough ram slots and get enough ram for your needs now but leave some slots free for expansion later. Also, get as many PCIe slots as you can afford, you'll need at least one for an additional network card now. If you want to run a Windows VM for games you'll need an additional Graphics card for pass-through leaving the integrated GPU on the CPU for the proxmox host.
Down the line, you may want to add a HBA card to expand your storage capabilities so that's another PCIe slot you'll need.

PCPartPicker Part List: https://uk.pcpartpicker.com/list/rbnRqp

CPU: AMD Ryzen 3 3200G 3.6 GHz Quad-Core Processor (£186.00 @ Amazon UK)
Motherboard: MSI X470 GAMING PLUS MAX ATX AM4 Motherboard (£79.98 @ CCL Computers)
Memory: Corsair Vengeance LPX 32 GB (2 x 16 GB) DDR4-3200 CL16 Memory (£114.42 @ CCL Computers)
Total: £380.40
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2021-01-04 16:33 GMT+0000

 

[WhiteTiger]

Down the line, you may want to add a HBA card to expand your storage capabilities so that's another PCIe slot you'll need.

Thanks for your reply.
What HBA board did you get?
To connect other SATA drives?

 

[bobmc]

I tend to pick them up on ebay, used dell H310's or IBM M1015's are readily available. Try to get them where they've been flashed with 'IT Mode' firmware - so that they behave like a simple HBA rather than a RAID-controller, which is what you want for running ZFS. You can flash them yourself but it saves the hassle if you can find them already done.

 

[WhiteTiger]

I am almost determined to buy a Ryzen 9-3950x, with 16 cores.
Should you think about overclocking?
What kind of liquid sink should I be thinking about?

 

[Dunuin]

I am almost determined to buy a Ryzen 9-3950x, with 16 cores.
Should you think about overclocking?

Servers should be as stable as possible and overclocking makes them more unstable. I wouldn' do that.

What kind of liquid sink should I be thinking about?

I wouldn't buy a All-In-One liquid cooler if the case got enough room for a big Tower sink. Cooling isn't better and they are more prone to failure.

 

[guletz]

A VM is for the pfSense firewall to manage a DMZ.

At anytime, a software/VM can have some problem, so in this case your DMZ can not function as you expect.
So IMO I would buy a cheap HW router, who is more relaible compared with a VM Firewall. For example a "Mikrotik HEX S".
Also, a HW router is faster compared with a VM firewall.


Good luck / Bafta !

 

[bobmc]

At anytime, a software/VM can have some problem, so in this case your DMZ can not function as you expect.
So IMO I would buy a cheap HW router, who is more relaible compared with a VM Firewall. For example a "Mikrotik HEX S".
Also, a HW router is faster compared with a VM firewall.


Good luck / Bafta !

pfSense works perfectly well as a VM firewall for several dozen users but I agree it can be an issue if your proxmox host needs maintenance or has a problem.

 

[Dunuin]

pfSense works perfectly well as a VM firewall for several dozen users but I agree it can be an issue if your proxmox host needs maintenance or has a problem.

It is possible to run OPNsense in HA configuration. I also thought about getting a second NIC and add that to my FreeNAS server so I could run two OPNsense-VM on different hosts so not every device is offline because of missing routing if the Proxmox server will stop working.

 

[AllanM]

Hi WhiteTiger,

Some suggestions:

1. Use server hardware for server purposes. You want a machine that can run for weeks or months between maintenance reboots without any compute or memory errors when hosting lots of VM's, and you also want something that can hold a lot of memory, which is something consumer CPU's with non-ECC unregistered memory handle poorly or don't support at all, since there is no electronic buffer to take the load off the memory controller.

2. Don't overclock a server.

3. Don't use pumped liquid cooling for a server, it's a fad for tiktokers. It is not a professional/reliable cooling solution.

4. I would expect to need no more than 2-4 ethernet ports in this sort of deployment. 1 should be a "WAN" link to your edge device, 1 should connect to your local network aggregation switch and carry all local vlans including management vlan. An optional IPMI interface to remote manage the underlying hardware (found on server hardware) and optional dedicated management interface for proxmox instead of carrying that management vlan on the local lan link. I run 7 network interfaces on the proxmox nodes in my home cluster, but 4 of those interfaces on each node are dedicated to cluster/ceph networks, and 1 is IPMI, so it's really just a LAN link with a bunch of VLANs and a WAN link to the edge modem. This arrangement works fine and would only be 2 interfaces if it were a single node without IPMI.

5. Ryzen 9 3950X is a great CPU, but almost certainly overkill on the compute performance and underkill on the IO and memory support for a good server.

What sort of budget are you aiming at?

A really good "server grade" (but tower form factor) platform to do this with would be a refurbished Dell Precision 7600 or 7900 series. These can be configured to hold up to 8 externally accessible 2.5/3.5" drives, and can hold up to 512GB of registered memory, and have lots of expansion slots. They are enterprise grade hardware that tends to be well supported by linux/unix distros and hypervisors. The 7600 series are older, Sandy/Ivy Bridge on DDR3, but they work fine for home and small business servers and are really good value. The 7900 series are Haswell/Broadwell on DDR4.

Here's a (2x8) 16 core machine with 256Gb RAM for under $1000: https://www.newegg.com/dell-precision-t7610/p/1VK-0001-4C758

Here's the same machine but configured with (2x6) 12 faster cores for under $1100: https://www.newegg.com/dell-precision-t7610/p/1VK-0001-4CWK8

I think I'd spring for the faster 2 X 6 core for this purpose. You'll get better real world performance out of the faster clocked CPU's for this sort of application. That system would have comparable compute performance to a more modern Threadripper 1920X.

 

[AllanM]

At anytime, a software/VM can have some problem, so in this case your DMZ can not function as you expect.
So IMO I would buy a cheap HW router, who is more relaible compared with a VM Firewall. For example a "Mikrotik HEX S".
Also, a HW router is faster compared with a VM firewall.


Good luck / Bafta !

If you're familiar with reloading a firewall config from backup, you know that very complex configurations with numerous packages installed often don't recover properly, so bare hardware is not a great place to configure a complex firewall unless you can afford to lose the config. I like a true hardware "edge" appliance for the peace of mind it provides (helps protect from vulnerabilities on a hypervisor and misconfiguration mistakes on the server cluster), but I don't like to run it with a complicated configuration.

The beauty of virtualizing a firewall is that it you can very quickly generate backups/snapshots of the complete firewall VM and recover back to a working state when applying complicated configurations/changes that might break something. At work our virtualized firewall has complex firewall rules between about a dozen vlans, IDS/IPS, DNS/IP filtering, several vpn server instances, and multiple CA's with dozens of certs for internal systems and vpn clients. It's nice knowing that we can just make a snapshot before making changes and move on without worry of permanently breaking the firewall.

Also, in a well designed clustered environment, the virtualized firewall is not dependent on the functionality of any single node or any single piece of hardware within a node. It can be migrated from node to node if there are hardware problems on a node, making it MORE reliable than a bare hardware firewall.

There is a performance hit, but so far we're able to saturate 1Gb between networks and get up to ~200Mbps on openVPN, which isn't far off from most bare hardware so the performance sacrifice kind of doesn't matter.

 

[guletz]

If you're familiar with reloading a firewall config from backup, you know that very complex configurations with numerous packages installed often don't recover properly, so bare hardware is not a great place to configure a complex firewall unless you can afford to lose the config

Hi,

It depens at what hardware Firewall do you use/have in mind. I use Mikrotik for many years(now I have around me about 90-100 pcs), and I do not face this problem even for a single time. Also in the case of very complicated configurations(hundred of firewall roules, tens of thousand IPs in no-access list, vpn, and so on) the recovery was succesfuly , without any problem at all.


Good luck/Bafta !

 

[guletz]

Also, in a well designed clustered environment, the virtualized firewall is not dependent on the functionality of any single node or any single piece of hardware within a node. It can be migrated from node to node if there are hardware problems on a node, making it MORE reliable than a bare hardware firewall.

It is also possible with any Mikrotik device!

Good luck / Bafta !