HDD usage option.

F

freebee

Well-Known Member
May 8, 2020
73
5
48
42
With the increase in SSD/NVMe memory prices worldwide, this type of memory tends to significantly increase the cost of backup infrastructure.
Is there any possibility that the Proxmox Backup Server project could rethink the use of HDDs and design a solution for better performance in this scenario? In the case of the datastore, defining a portion of the storage for smaller SSDs (backups with retention of only 1 or 2 most recent backups) and another portion for HDDs, increasing the retention period. The same datastore could accommodate both types of memory.
 
IIUC what you propose: that makes little sense and that setup you describe can be accomplished with local sync jobs (SSD datastore where backups are done, then sync them to HDD for "archival").

The performance issue with HDDs on PBS isn't backup or restore themselves (which of course would be faster with all-flash datastore), but mainly with Garbage Collector and to a lesser extent with Verify. If you store most of your snapshots on HDD you will have to run GC on them too, so even if some of your backups are on SSD you'll have GC performance issues anyway. Check ZFS special device to get the best of SSD+HDD on PBS [1]

[1] https://pbs.proxmox.com/docs/sysadmin.html#zfs-on-linux
 
As I always say, just don't use ZFS on a PBS spinning rust datastore.
For example a small datastore (750GB) on BTRFS just has a GC time of 7 seconds.
The verify just needs longer than on SSDs ofc.
 
JensF said:
For example a small datastore (750GB) on BTRFS just has a GC time of 7 seconds.
Click to expand...
Because it has just 750GB and I bet that all metadata is already cached. Try rebooting the server and running GC again or try running GC on a 140TB datastore on BTRFS. I've done such tests and performance is similar to ZFS. Not to mention that you can't have something like RAIDz on BTRFS in a supported way.
 
Updates installed and rebooted. GC time 25 seconds.
And yes, I tried every existing file system. BTRFS were always faster.
The bigger the store is, the longer it takes. That's true.
 
You must log in or register to reply here.
Top Bottom