PBS hardware requirements

[f4242]

Hello,

I'm looking for hardware to host two PBS servers. I read some posts where the author strongly suggest using only entreprise SSD drives. Even our prod still use HDD most of the time, SSD are still the exception in our infra, so buying SSD for storing backups appear very strange for me.

What I need to backup :
- 7 PVE hosts (44 VMs) on site
- 3 PVE hosts (27 VMs) remote (sync job from the other PBS)
(vice-versa)

What I think to use as PBS server hardware:
- Dell PowerEdge R720XD
- CPU: 2x Xeon E5-2667 V2 3.33 Ghz
- RAM: 128 GB
- Hard drives: Western Digital Red Pro 12TB
- Raid controller: Dell PERC H710 Mini Raid Controller 6Gbps with BBWC
- Raid configuration: 8 drives, RAID 10 (hardware RAID, no ZFS, using only ext4 FS)

The server would run Proxmox VE and PBS inside a VM. Some others smaller backups job would run in another VM (like database dumps, file system backups).

Is it really wrong to use that kind of hardware with Proxmox Backup Server?

Thanks.

 

[dcsapak]

Is it really wrong to use that kind of hardware with Proxmox Backup Server?

it's not 'wrong' to use it with pbs, but you should have the proper expecations. pbs works by splitting the backup in chunks (~2-4MiB mostly), so things like garbage collection,verify, etc. must access many small files, which depends more on for performance, and hdds don't provide the level of iops ssds can

personally i'd test it with the current configuration (if you already have the hardware) and see if you are satisfied with the performance

this largely depends on how often and how much data you are backing up (and how often you do garbage collect, verify, etc.)

 

[UdoB]

The server would run Proxmox VE and PBS inside a VM

You may install PBS on the PVE-Host side-by-side. If this is a "nearly dedicated" PBS that's what I would do. This completely avoids the problem to get the storage into the VM.

(Actually I do have the same hardware (R720 + H710Mini) in my Test-Cluster but I am a fan of ZFS and so I went the "reflash to IT-Mode" Road...)

Good luck

 

[Dunuin]

You should at least use ZFS so you can use a SSD for metadata which will speed up GC by magnitudes. So for example 8 HDDs in striped mirror (raid10) for data + 2 or 3 small enterprise SSDs in mirror for metadata:

Recommended Server System Requirements¶

• Backup storage:
  • Use only SSDs, for best results
  • If HDDs are used: Using a metadata cache is highly recommended, for example, add a ZFS special device mirror.

8x 12 TB in a raid10 means your datastore will hold around 24 million chunk files while doing something like 400 IOPS. A GC task for example will have to read and write the metadata of all those 24 million files, so atleast 48 million IO. You can estimate that yourself: 48,000,000 IO / 400 IOPS = 120,000 seconds = 33 hours. So a daily GC for example won't work.

 

[f4242]

Hello,

Thank you for your answers. I was busy on others project and I'm now back on the PBS project!

The max storage for one server I plan for the VM dumps with PBS would be around 5 TB in the future (so not 24TB of storage dedicated to PBS). Daily GC would take around 7 hours if I understand well the last comment from @Dunuin.

One alternate option I think about is using a mix of SSD and HDD. SSD raid array would be used only for PBS and HDD array for others backups. I wonder how consumers grade SSD like Samsung EVO 870 would work with this workload? I would use 4x4 TB in RAID10, so 8TB of usable storage. This specific model is rated at 2400 TBW with a 5 years warranty. If I expect to right less than 1tb/day, I guess I'm fine for at least the 5 next years?

 

[Dunuin]

Keep ZFS write amplification in mind. I got an average write amplification of factor 20, so 2400TB TBW would be reach here after just writing 120TB of data.

 

[f4242]

Keep ZFS write amplification in mind. I got an average write amplification of factor 20, so 2400TB TBW would be reach here after just writing 120TB of data.

Wow! Thank for the hint. Another reason to not use ZFS for this server I guess haha.

 

[DC-CA1]

Keep ZFS write amplification in mind. I got an average write amplification of factor 20, so 2400TB TBW would be reach here after just writing 120TB of data.

do you think this would occur on RAIDZ1 as well ? or you refer to something like RAIDZ2 ?

 

[Dunuin]

do you think this would occur on RAIDZ1 as well ? or you refer to something like RAIDZ2 ?

ZFS is general. But shouldn't be that bad with PBS, as there isn't that massive amount of small sync writes. But still way more then other filesystems would do which don't care that much about data integrity.

 

[mow]

But still way more then other filesystems would do which don't care that much about data integrity.

Integrity checks shouldn't need to write that much, though.

 

[Dunuin]

Integrity checks shouldn't need to write that much, though.

Have a look at how Copy-on-Write works. And for redundancy, ZFS for example is writing 3 copies of metadata. And sync writes will be written twice. First to the ZIL, then later again the final records. This of cause all got overhead.

 

[mow]

Have a look at how Copy-on-Write works.

It should still need less writes overall than a fs that doesn't do that and therefore needs to write everything multiple times.

And for redundancy, ZFS for example is writing 3 copies of metadata.

That should only be a problem with mostly tiny files where the data is in the same order of magnitude as the metadata.

And sync writes will be written twice. First to the ZIL, then later again the final records. This of cause all got overhead.

Not much different from e.g. ext4 with full journal. (But admittedly one of the reasons almost nobody does that.)