Dell Server Hardware Advice

[helojunkie]

Hello Everyone -

We currently have a 4-node cluster running on Del R820s, each with 4 x E5-4650 2.7Ghz CPU, 256GB RAM, 10GB NICs, and mirrored SSD drives for VM/CT and NFS for other storage. We run Cisco Nexus 10G switching as our backend corosync separately from our 10G frontend.

These servers were purchased used about three years ago and have done a great job for us, but we are starting to load them down, and I want to start augmenting them (for eventual replacement) with some more powerful Dell (preferably 1U) servers. Right now, I would like to purchase 2 x 1U servers to replace one of my R820s. Same overall footprint in the rack but a lot more capability.

Unfortunately, there are many Dell servers out there that range from inexpensive to over-the-top expensive and everyone selling them wants me to buy the biggest and best.

I was hoping someone here has experience with Dell servers (specifically) and what would be a more powerful option in 1U than my current R820s that run Proxmox without any issues.

I appreciate any guidance and wisdom.

 

[jdancer]

If buying used, I prefer 13th-gen Dells, specifically the R730xd in which you can use the rear drives as RAID-1 OS boot drives.

If looking for something else newer, then 14th-gen Dells, like the R740xd.

I would not get 15th-gen Dells until the Linux drivers for that hardware has matured.

 

[TonyR]

I am a little late to the party but here's my 2 cents.

I *love* the R620 / T620 range and here's why.

The R620 is 1u and where I am using colo that keeps the costs down per U. The 620 range in itself is really respected and tested over the years with great support in general and a reputation as a solid platform with good spares availability and selection of component parts.

Generationally, the performance difference between the. 620 and 630 in the real world is not much whilst the cost difference is substantial. You can get a loaded R620 for a couple of hundred £/$/Euro. The 630 supports more cores at a lower clock speed and DDR4. Also, Bifurcation and NVME native but that's no big issue.

The processor of choice is the E5-2690 V2 - that's 10 cores at 3GHZ. That for me is a sweet spot between single core performance and multi core performance. Depending on the workload, some apps still love single core although multi-core aware apps are becoming more common. You can pick up CPU's for £10 a piece and a cheap dual CPU chassis for £40.

Next, RAM. This box will take LRDIMM - that's load reduced and it will only run in servers so is cheap second hand. I recently picked up 512GB in 32GB sticks for £100. Now we have a half terrabyte server with 40 threads for less than £200. The 620 will actually take 768GB in 32GB sticks or 1.5TB in 64GB sticks. Anyway, DDR3 is CHEAP and it will run at either 1600 or 1333 depending in the config you pick. It's plenty fast enough.

Lastly, keep it all solid state unless you need vast storage. I use 3.2TB NVME/PCIE VNAND cards which are enterprise level with stupid TBW levels and performance. You can pick these up new for £100 to £150 as only us types know what they are. The chassis will take 6 as I recall.

You can flash the onboard 710p to IT mode easily enough if you need to boot from a SSD or HDD and always have the option for quite a few 2.5 SSD's in the front or 5TB HDD's if you need a 40TB ZFS raid.

All in all, the 620 in this config is a total beast. You can also get Dell/Intel 1g/10g LOM cards for £30 so native 10G on board. I pass these through to PFsense VM's and then force all traffic through that firewall. One LAN, one WAN (10G) and 1G for cluster.

I find my VM limit tends to be the amount of disk or memory I have as nothing seems to scratch the CPU's

For quiet office/Bedroom use the T620 is the go-to. As a tower, you can load it with 8 full height PCIE cards and it is whisper silent. If it is not quiet enough you can add the optional front fan tray and it will run at full load less than 40db noise.

I hope that's helpful. I have been looking at the performance versus economics of this for a few years and even in 2023 it's hard to justify going to a newer generation as there's nothing really I need specifically in the new hardware and for me it always comes back to the 630 requires all new ram (DDR4) and the CPU performance of the 2690-V2 is the same or better for single core clock speed and multi-core performance than anything the 630 offers.

Power wise they are about the same to run and the capital outlay, spares cost and cost of operation in hosting wins out for the 620 every time.

 

[helojunkie]

This is great information, thank you both. I would really like to move to 1U servers to save space in my computer room so I think the 630/630 is inexpensive enough to give a try. I just don't know how powerful they will be compared to my 4 x CPU R820s. Do I need two of them to replace a single R820?

 

[TonyR]

This is great information, thank you both. I would really like to move to 1U servers to save space in my computer room so I think the 630/630 is inexpensive enough to give a try. I just don't know how powerful they will be compared to my 4 x CPU R820s. Do I need two of them to replace a single R820?

Who knows. That depends on the config of the server and the workload it is doing. Check the current CPU usage - I suspect it isn't scratching it much.

The biggest thing in performance terms would be memory and NVMe. The IOPS on VM workloads is orders of magnitude better with that over SSD which in turn is way better than SAS or SATA disks.

I suspect with what you said above the SSD is the bottleneck. 10G is overkill for corosync. You will be just fine with 1G ports. What is your workload? A lot of VM's or a few VM's with heavy use?

Things like Database will be better with more RAM (If the DB will fit in RAM) or NVME for concurrent read/writes on heavier workloads.
Things like web servers and PHP like single thread performance over multi-core CPU's. Switching to OpenLitespeed for example can really reduce web served high concurrency workloads.

Lots of VM's will love NVME due to the high IOPS - it can do a ton of concurrent read/writes without slowing down compared to SSD.

All of these things are not CPU dependant particularly. Also, if you have a load of Prox VM's on the same virtual network (not physical) then there will be virtually zero bottleneck on the LAN side of that. It's only when you go box to box you need the LAN, and even then unless you are transferring hundreds of GB then 1G will be enough. It's suprising how much you can shift over a 1G link. A single 4 port 10G card can hook five boxes together without a switch and a single 1G card can provide your internet. I doubt you will need more than 1G to the net.

Back to the question, who knows. But I suspect the answer will be you don't need 4 CPU's unless you are doing something crazy computationally.

If you go the 630 route there are also E5-V4 CPU's upto 22 cores so 2 CPU's will easily have the same oomph as your 4 existing ones if really necessary.

Again, maybe check the CPU load on the prox summary screen and see where you are at

 

[TonyR]

This is great information, thank you both. I would really like to move to 1U servers to save space in my computer room so I think the 630/630 is inexpensive enough to give a try. I just don't know how powerful they will be compared to my 4 x CPU R820s. Do I need two of them to replace a single R820?

Oh, I forgot to say

I have tried the 620 with both RAID ZFS SSD (8 x 500GB) and a single NVME.

The difference is night and day with the NVME. Even simple file copies take fractions of a second compared to tens of seconds for the SSD array.

 

[helojunkie]

Oh, I forgot to say

I have tried the 620 with both RAID ZFS SSD (8 x 500GB) and a single NVME.

The difference is night and day with the NVME. Even simple file copies take fractions of a second compared to tens of seconds for the SSD array.

All my systems use only NVMe Drives (Intel DC4500 series) in a mirrored ZFS configuration. I also have a PBS as well so I am not backing up locally. Storage of ISO, etc is done via NFS. The servers don't break a sweat, I just want to downsize from 7 or 8 year old servers to something newer and smaller.