Backup tool recommendation?

[Dunuin]

Hi,

I want to change my backup strategy but I'm not sure whats programs to use. I thought maybe you guys got some recommendation or experiances to share with me.

I've got a FreeNAS server running 24/7 with NFS/SMB/rsync/FTP services. That FreeNAS server is using snapshots, raidz1 and is replicating everything weekly to another FreeNAS server. So stuff stored on that NAS using NFS, SMB and so on should be quite save.

Right now I backup my Proxmox VMs by using vzdump and storing the vzdump files on the NAS using SMB. That is working fine but takes alot of time because vzdump doesn't work incrementally. So one idea was to install the proxmox backup server somewhere. If I get it right I can install the PBS somewhere, add my NAS using NFS/SMB as a storage and use the BPS to do incremental backups of all VMs? Or do I need a special type of storage to be able to do incremental backups (like ZFS)?
FreeNAS got KVM virtualisation using bhyve so I could run a PBS VM on the FreeNAS server so that PBS is usable even if the complete proxmox server fails.

The second problem I got is the backup of several Linux, Android and Win10 hosts. Right know I use rsync clients to backup (one way sync) selected folders to the NAS. That is working fine for Android but I don't like the Win10 clients. As far as I can see there is only DeltaCopy available for Windows that is running as a service. That is basically working but often there are problems (for example paths to long to be stored on the NAS) but there is no way to get notified if such errors occure and no easy way to check/verify what is acually backupped and what not. I want to sleep worryless without manually checking the rsync logs every day line by line if something wasn't copied.
I tried other programs like PureSync that got a nice GUI with notifications and so on but these are not using rsync and it takes forever to do backups. And if I backup some TiB and millions of files most programs like PureSync just can't handle the amount of files and crash or get unresponsive. The only twi I found so far were robocopy and DeltaCopy that can handle this.

Is there maybe a good tool that can handle millions of files, works with Win10, linux and android and that got a nice client that keeps tracks of everything and will warn me if something isn't backed up?

Setting up a VM as a server to backup stuff to the NAS wouldn't be a problem. I read a bit about bacula, borg and so on but these look a little bit overkill because ZFS is already doing compression, versioning using shadow copy and snapshots, replication. And I like that I can just access all backups using SMB if something goes wrong.

For the backup of the system disks of all hosts I would like to use clonezilla to create block level images that I can write to a NAS share.

Would be great to hear how other people are doing the backups.

 

[fpausp]

Would be great to hear how other people are doing the backups.

I use PVE (HW-Raid 10 - 8HDDs) to run my VMs and a PBS (ZFS Raidz - 4HDDs) as a backup-target to be able to backup my VMs incrementally. Additionally on that PBS-Server I have NFS running for vzdumps and Urbackup for my Windows 10 Clients. Hopefully there will be soon a proxmox-backup-client also for Windows Clients/Servers. For Linux Clients (Debian, CentOS, Arch,...) you can also use the proxmox-backup-client.

 

[Dunuin]

That UrBackup sounds nice. I think I will try that for my Win10 hosts. I wasn't able to find hardware recommendations. How much CPU/RAM do you think I need to backup 3-5 hosts?

 

[fpausp]

How much CPU/RAM do you think I need to backup 3-5 hosts?

This is just one Client and the other stuff...:


This runs on a HP Microserver Gen 10.

 

[Dunuin]

@fpausp:
Do you know if it is possible to manually add or block urbackup clients? I've added 2 subnets to my urbackup server VM (1Gbit + 10Gbit) and I got some clients that only use the 1Gbit subnet but one client got a NIC in both subnets. Thats a problem because that client is now using the 1Gbit NIC and not the fast 10Gbit NIC. So I need something to tell the server or client not to use the 1Gbit subnet for specific clients.

Edit:
Looks like blocking Discovery for that host using Proxmox firewall rules worked.

 

[fpausp]

Do you know if it is possible to manually add or block urbackup clients?

Sorry, screenshot just in german... You can use different values for each client:

 

[Zozfer]

However, I suggest urbackup too, but maybe you would take a look at Veeam Backup , Duplicati and Gs Richcopy 360, great features are there .

 

[Dunuin]

I had to remove urbackup again. The problem was that most of the time the backup crashed sooner or later and a crashed backup still was kept and counted as a backup. So if I for example setup to only store max 5 backups and 4 of 5 failed, I only got one complete and valid backup. And as far as I know a failed backup can't be resumed. A problem also may be that all my drives are encrypted by veracrypt and urbackup can't snapshot encrypted drives so it can't backup any files that are in use.

 

[Zozfer]

I had to remove urbackup again. The problem was that most of the time the backup crashed sooner or later and a crashed backup still was kept and counted as a backup. So if I for example setup to only store max 5 backups and 4 of 5 failed, I only got one complete and valid backup. And as far as I know a failed backup can't be resumed. A problem also may be that all my drives are encrypted by veracrypt and urbackup can't snapshot encrypted drives so it can't backup any files that are in use.

So check with the support of Gs Richcopy 360, I think they have options to resume the backups and to back up the files in use .

 

[Dunuin]

Me again. I just did a Upgrade from Win10 to Win11 and while doing that also switched my disk encryption from VeraCrypt to Bitlocker. The UrBackup FAQ only says that backups of opened files isn'T working with unlocked Veracrypt/TrueCrypt volumes because VeraCrypt doesn'T support the use of shadow copies. I now hope that shadow copies will work with bitlocker so I could finally use backups...

I looked for backup tools again and Veeam and UrBackup looked promising.

My idea was to store the Backups on a SMB share on my TrueNAS server using zstd compression. But I'm not sure if I would like to enable ZFS deduplication for that pool.

Someone knows if UrBackup or Veeam Windows Agent is capable doing deduplication on file level like PBS? I've read that UrBackup is using hardlinks and symlinks to save space what sounds it would work like some kind of deduplication. But is that only deduplicating complete files or also smaller parts of bigger files? Wouldn't be very helpful if I would backup a 100GB file, change 1GB of that file, back it up again and deduplication using references wouldn't work anymore because only 1 GB changed so 200GB instead of 101 GB would be needed.

And for Veeam it is hard to find out/understand how it works under the hood. As far as I understand it the Veeam Windows Agent isn't running any server and will just store stuff on my SMB share. And it sounds like it is only storing files/folders as big image files. So I guess there is no kind of deduplication and storing 5 backups of identical data will use 5 times the storage? And I found out that there is a server that can be used with Veeam Windows Agent that is called Veeam Backup & Replication but that only runs on Windows (which I don't really like because WinVMs consume so much Memory and CPU and can't use ZFS) and it looks like this is primarily made for backing up VMs. But this server offers datastores like PBS and I've read that these datastores can use deduplication but I wasn't able to find out if that is done in software or if it has to be done on storage level.

What would you recommend to incrementally backup around 1TB of data in different folders stored on different bitlocker encrypted drives if I want to store them on a SMB share and keep them for some months without that this uses multiple of TBs of backup storage?
I really like how PBS handles this (storing 17.28 TiB of backup snapshots of 200GB of VMs and only using 400GB of storage) and I hoped to find something that works similar but is working for windows too. Too bad that it isn't planned to add Win support to PBS.

 

[Dunuin]

So I looked at many backup solutions...

1.) Veeam Windows Agent
That looked really promising but there is a big problem...
Without the "Veeam Backup & Replication" running as a server (which will only run on Win and would need a WinVM with atleast 8GB of RAM that I don't want to sacrifice) the agent needs to use SMB shares or local disks. If my Windows hosts got write access to that Backup SMB share so that the Veeam agent can use it, that also means that ransomware can delete/encrypt my backups so the backups are totally useless unless I enable snapshotting of that SMB share with a very long snapshot retention (for example 2 months). But Veeam is doing a lot of writes and these writes won't be deduplicated in software so if I want to backup 1TB of data my ZFS dataset storing the backups will grow to something like 12TB.

2.) UrBackup
UrBackup got a proper server client model (so fine for ransomware protection without snapshots) but it only deduplicates using links and not chunking. So only complete files can be deduplicated. Lets say I backup my game folder and there is a game with a 50GB file. Now steam patches my game and some MB of that 50GB will change. Now UrBackup can't deduplicate this file anymore and it will consume 100GB instead of 50GB.
So that isn't space efficient neigher.

3.) Restic
Restic sounds really nice but it is still in early developement and there is no windows GUI/service I would like to have. I have no problem setting things up using CLI but I atleast want some kind of app sitting in the taskbar showing me backup progress and alerting me if a backup fails.

4.) Borg
Sound really nice too and sounds like it would do everything I want it to do except that it is not made to backup Windows hosts.

5.) Bacula
This looks great but installing and maintaining it sounds super complex and time consuming. I just need to backup 2 or maybe 3 Windows hosts and Bacula sounds like I would need to spend one hour per weeks to keep it updated and running.

6.) Duplicati
Version 2 sounds like it would do the job (chunk deduplication and VSS support) but again it is client only. So no protection against ransomware. I really don't get why so many of these backup tools only work as a client. My idea of a good backup solution would be that I got a client that can only write but not prune or edit backups. And then a server on another machine that will prune and delete these backups. So backups are save and protected against manipulation.
Need to look if Duplicatis deduplications works fine. Maybe it is still an ransomware safe option if duplicati isn't writing the same data again and again so I can enable long snapshots retentions to be able to rollback the whole SMB share.

7.) Rsync
There is just no useable rsync client for windows. DeltaCopy and cwRsync are very old Rsync clients running super outdated rsync versions causing compability problems with the Rsync server my TrueNAS is providing. And Rsync can't backup opened files because both of these doesn't support VSS. There is one modified version of cwRsync that got VSS support but that one isn't updated for 13 years...

8.) BackupPC
Here the deduplication sounds nice too, but this tool is server only. There is no client. The server is grabbing its data from the hosts you want to backup and it sounds like it is doing this using SMB shares. I'm not sure if VSS is working then.