A New (first) Proxmox Backup Server Setup Questions

[pd_techteam]

Hello Guys, first let me tell you im very much surprised how good proxmox is (only used Ubuntu Server and Unraid before, but proxmox is (for me) levels above them).

Hardware:
Hetzner Root Server
AMD 3700x
64 GB ECC RAM
2x 1TB SSD (NVME)
4x 22TB HDD

What do i want:
I want to install an Offsite Proxmox backup Server for a lot of Data (in the future), i do backups on my local internal PBS running on my PVE and want to use the hetzner Server as an Offsite Backup. Later have the possibility to Backupd directly to the PBS.

Current Plan:
- Sync Local Backups to the PBS on Hetzner.
- Create Backups over the internet directly on the PBS
- 2x 1TB SSD in ZFS Raid 1 Mirroring
- 4x 22TB HDD in ZFS Raid Z2 for double parity
- Mirror SLOG, L2ARC and Special Device

Setup Questions:
- How big should the SLOG be?
- How big should the L2ARC be?
- How big should the ZFS Special Device be?
- SLOG, L2ARC and Special Device on the SSDs or HDDs?
- The Datastore will be on the 4x 22TB HDD Array, what should i do with the leftover from the SSD (if any)

If my PBS dies (the os itself) how do i restore then?

I appreciate any Help

- Nils

 

[omgs]

I tend to believe one should have a "small" PBS ct in PVE, and from then on, have as many remote replicas you just want. In PVE, have a "local" storage and another storage for the PBS. The idea is to backup the PBS ct to the local storage "when needed" (this backup should be less than 2GB), and the backups for your cts and vms go to the PBS. You may need a transition if you have many backups in a non-pbs storage.
So, in the PBS ct, have separate mps for the system and the datastore(s). This way, you can just backup the rootfs (i.e, have mps excluded from backup). You can backup the ct locally (less than 2GB) and also to the PBS itself (i.e, one in a local storage and a second backup in a PBS datastore that you can sync to other servers). This way, if something happens to the PBS OS, you can restore the ct, and the mps will mount themselves automatically.

I'm using this approach in several sites and I find it very comfortable.

 

[VictorSTS]

- 4x 22TB HDD in ZFS Raid Z2 for double parity

IMHO, makes no sense at all using RAIDz with four drives, use RAID10: you get the same amount of accessible storage and will get up to twice the performance, among other things. Yes, even with the benefit of RAIDz2 supporting losing any two drives without dataloss vs RAID10 losing 2 drives of different mirrors.

- How big should the SLOG be?

Zero. A workload like PBS does very few sync writes. Using SLOG provides no advantage.

- How big should the L2ARC be?

Zero. Three reasons:
- As a minimum. you should have around 1GB of RAM for each TB of ZFS storage (~80TB in your case). Your server is already undersized ram-wise.
- L2ARC requires memory for it's operation which you don't have to spare.
- L2ARC provides very little benefit in a PBS workload unless it is huge (tested on ~84TB HDD storage + 4TB NVMe L2ARC). If I have to increase the NVMe storage beyond that, I would just use it for PBS datastores directly.

How big should the ZFS Special Device be?

For a PBS workload, around 0'5% of your usable storage provides ample space with special_small_blocks=16k. Be sure to enable small_blocks on PBS datasets only. Place special device on NVMe drives, of course.

what should i do with the leftover from the SSD (if any)

Use it as swap, give more space for /, use it as storage for small local VMs...

If my PBS dies (the os itself) how do i restore then?

What do you want to restore exactly?