Strong Datacenter using Proxmox

[eduardinho21]

Iam looking to build a "good" Datacenter using Proxmox.

Hardware I have:

2x HPE DL360gen9 | 64GB RAM | 4 x 4 TB HDD | Intel 12 core total

1x HPE DL360gen8 | 32GB RAM | 2 x 2 TB HDD | Intel 20 cores total


With it, Can build an Cluster with “high availability” and backup server?

My 1st idea was, 1x server for Prodution, 1x server for “fail over” or Disaster recover, and 1x server for redundant Backups.

 

[LnxBil]

My 1st idea was, 1x server for Prodution, 1x server for “fail over” or Disaster recover, and 1x server for redundant Backups.

This would work, but why would the backups be redundant? For this setup you would need ZFS and ZFS requires direct disk access, so no hardware raid.

"datacenter" is a bit exaggerated for just 3 machines and no real high availability.

 

[eduardinho21]

This would work, but why would the backups be redundant? For this setup you would need ZFS and ZFS requires direct disk access, so no hardware raid.

"datacenter" is a bit exaggerated for just 3 machines and no real high availability.

Hello LnxBill,
i touched the other Big question:
ZFS vs Hardware Raid. If i have an RAID controller why not?
I wich redundant backups to recover from hardware damage or ransomware atack.

 

[ph0x]

Zfs is superior, full stop. But it needs - to benefit most of it - access to the disks. So you either use a HBA or you flash your RAID controller to IT mode.
Your setup is not HA in the usual sense. HA starts with three nodes, which would also be possible in your case. Remember that you cannot even restart a node with only one productive server. And failover in a cluster without a shared storage is only possible when replicating all disks all the time. With a RAID controller and therefore (usually) local storage, there's no failover.

 

[eduardinho21]

Zfs is superior, full stop. But it needs - to benefit most of it - access to the disks. So you either use a HBA or you flash your RAID controller to IT mode.
Your setup is not HA in the usual sense. HA starts with three nodes, which would also be possible in your case. Remember that you cannot even restart a node with only one productive server. And failover in a cluster without a shared storage is only possible when replicating all disks all the time. With a RAID controller and therefore (usually) local storage, there's no failover.

So in this case, What you you would recomend?

 

[ph0x]

I would look for some decent SSDs to begin with. VM images on HDDs can be gruesome. But that of course depends on your workload.
You can indeed have one main server which replicates to the second and have a third one for backups. You then need the RAID controllers to work in IT mode and remember that this is not true HA, but still very HA.

 

[eduardinho21]

I would look for some decent SSDs to begin with. VM images on HDDs can be gruesome. But that of course depends on your workload.
You can indeed have one main server which replicates to the second and have a third one for backups. You then need the RAID controllers to work in IT mode and remember that this is not true HA, but still very HA.

Thank you. Yes, this is not a real HA, but still a good cluster config in my opinion too.
I can't buy Ssd's now but i don't need it. I Will replace 4 x 4TB HD's for 4 x 2TB, and put the 4TB HD's in PBS. What you think?

 

[Dunuin]

PBS also should use SSDs, as PBS needs IOPS performance. 4x 4TB of HDDs as striped mirror would give you 6.4 TB of usable space for backups. PBS will save everything as small chunk files of max 4MB in size (usually more like 2MB). So you probably will get around 3.2 million files that need to be read and written when doing maintance tasks. Lets say your raid10 got 200 to 400 IOPS and you need to read+write the atime of every file once when doing a GC. That you got 2x 3,200,000 IO / 300 IOPS = 21,333 sec = 6 hours for just a GC.
You at least should combine HDDs+SSDs for the PBS datastore. HDD-only will perform terribly.

And "decent SSDs" usually means Enterprise/Datacenter grade SSDs that are highly recommended for performance, reliability, data integrity and durability.

 

[LnxBil]

I can't buy Ssd's now but i don't need it.

Sorry, but I don't run any system (even the embedded ones) on only disks. I would never run any system without SSDs. Times are over. Used enterprise SSDs are very cheap and a disk system IS NOT up to any task nowadays. Just imagine a default OS start on disk vs. SSD and now multiply that by the number of VMs on your system. It'll be dead slow, especially if your disk head will constantly move around on your disk.