Hardware advice

[dredger]

Hello,

I came to this forum to ask for an advise for the selection of hardware. I hope it’s the good place for this kind of question(s).
I’m not experienced in proxmox and do not perform anymore hardware setup, but I still like to select the hardware that will fill our workstations or servers.

We have a proxmox server running since 2018, with only 1 permanent VM so far (Win2019 with TFS for development, only 3 devs), other VMs are run on-demand (for tests), sometimes we keep them running but they are dormant. In summary the server is not really charged. We’ll probably have a 2nd VM (still Win2019) running permanently before end of this year (very low charge too), and maybe one more in the future.

We recently had a problem which was solved but which pointed out that it would be safer for us to have a second physical server ready to take the hand if the production server fails.

Looking at the use of it (globally a low charge), I do not want this time to opt for a professional motherboard with 2x Xeon a ECC memory (which is the case for the actual server), just a more classical hardware + add an UPS and a redundant power supply.


As I like the MSI motherboard, I found one which looks nice on paper and would like to know if some here think it would fit for the job ?

I hesitate between 3 of them :
- The PRO Z690-A (https://www.msi.com/Motherboard/PRO-Z690-A)
- The PRO Z690-A DDR4 (https://www.msi.com/Motherboard/PRO-Z690-A-DDR4 )
- The X299 PRO (https://www.msi.com/Motherboard/X299-PRO)

I like the Z690 because it has 4x M.2 slots, and it’s more recent, but more recent hardware means potential instability, reason I came here to ask what experts think about.
Regarding choosing the DDR4 or DDR5, it’s a bit the same. I read that DDR5 is supported only since a short time by proxmox (it’s also quite more expensive than DDR4 but it’s, in this case, a detail)
The main reason I listed the X299 MB is because it can go up to 256Go of ram, but I do not think i would need so much memory before a long time. Second reason I listed it is because it’s older hardware and it may be more stable with proxmox (?)

About M.2 :
I noticed on the Z690 MBs that for the 4x M.2 drive, 3 of them are in 64Gbps and 1 is in 32Gbps. (let’s say that A, B & C are the 64Gbps ones and D is the 32 one)
My initial idea was to fit the server with M.2 drives only, 2x2 in raid, and avoid any other SATA disks, we do not need much disk space, but with this fact of 3 of them being in 64Gbps and 1 in 32Gbps I was wondering if the lone limitation of creating a raid with C & D will only result that this raid unit will be bottlenecked at 32Gbps … or if this could bring other problems ? (I do not think so but still wonder)

If any here has a better idea in how best allocate the disks/raids, please advise.
I see 3 main roles : the boot disk, the cache & logging disk and the storage (of VMs) disk.
What would be the best configuration ? Boot on one M.2 raid (in this case 2 small ones will be sufficient) and the rest on the 2nd M.2 raid ?
Or adding 2 SATA SSD (or 4) in raid for VM storage ?

Note that the 2 main VM would be backed up each day and the backup server would remain switched off, ready to be used in case of problem with the main server, This is the extra warranty I want to reach by setuping a second proxmox server.

For the rest I would fill directly with 128Go of ram (selecting quality brand/model), a redundant power supply and add an UPS.
For the CPU ... not the latest one, but a good one. If any has an advise about, please go on.
For the GFX, I do not see any reason to add a dedicated GFX card, if any think I should, please advise.
At least for the casing, I do not want yet a rack and hesitate to take a big tower or a normal one (the BeQuiet 700 or the BeQuiet 900 for instance). I’m wondering of the usefulness to take a big tower casing, it’s just looks now to me like an old habit which is not anymore much more needed.

That’s all. If any here has the knowledge but also the time to write down some advises then this would be great.
Regards

 

[dredger]

I up once the post, if no answer or advise i'll go on with the first MB listed and 4 M.2 disk.
About the MB, I was just wondering if there's any precise reason to avoid it or not.
About disks, i read in another post that they can be setup in a zfs stripped mirror and thus avoid the 2x2 raids i mentionned. But the "problem" (if it's one), or topic, of having one M.2 in 32Gbps remains questionable. Other questions or interrogations are of less importance but still, having an expert confirmation remains useful.

Nice week-end to all !

 

[Dunuin]

I would stick with proper hardware where the primary focus, when designing it, was reliability and good performance for server workloads and not fancy RGB or overclocking. Consumer hardware is fine for a homelab but I wouldn`t use it for a productive system where you rely on to earn money.

 

[oscaro]

Just on the topic of possible motherboards to purchase. ASRock Rack has good server grade mobos for sale. [0]

The X570D4U or X570D4U-2L2T is really good and there is a forum with tons of comments on the motherboard and their experiences with it. [1]

It has IPMI so you can remotely boot it over the network and do other cool things. ECC RAM works great. It has two of the fastest M.2 slots. (PCIe4.0 x4)

Would recommend!

[0] https://www.asrockrack.com/general/productdetail.asp?Model=X570D4U
[1] https://forum.level1techs.com/t/asrock-x570d4u-x570d4u-2l2t-discussion-thread/159996

 

[dredger]

I would stick with proper hardware where the primary focus, when designing it, was reliability and good performance for server workloads and not fancy RGB or overclocking. Consumer hardware is fine for a homelab but I wouldn`t use it for a productive system where you rely on to earn money.

Thanks for the advise.
As said, it's to strenghten the overall data security by having a 2nd proxmox server, and there's no particular urgency.
I followed the recommandation of a tech guru on another forum to build a proper station with reliant hardware, not searching for high power or speed but reliability, my own concern was to setup all using M.2 disks and quality hardware.
If, with no luck, Proxmox can't run on it or unkown random errors happen then this PC will be used whether as workstation or direct physical unix or windows server.
On the other hand, if it runs fine this will be a good thing to know and i can come here to post the hardware configuration. But as it's not yet fully determined i'll avoid to do it.

@oscaro i checked the link, lot to read. I do not know well ASRock and finally opt for an MSI MB, but another one than the ones i listed, with higher quality parts and a better VRM (voltage regulator module)

I'll maybe post back here in the coming months to report about, we'll see.
Thanks for your time.