Feature Request: Backup Staging Drive for SMR Capable Linear Writes

[jw6677]

Feature Request: SMR Capable Linear Writes

Loving what proxmox is up to with PBS!

One feature which would be immensely helpful would be the ability to safely use SMR drives without issue.
To be transparent, I simply have some I would love to use, but the use-case for enterprise is clear. Substantially lower backup storage expense.

As you know, SMR chokes on random reads and random writes, but is nearly indistinguishable, albeit slower, from PMR drives for long continuous writes.

As such, to address the use of SMR drives with PBS the feature of "Select backup staging path", a staging path to be selected.

For example, a single SSD drive, or conventional SAS HDD for all backups to first be written to initially, prior to final migration to the datastore, in one continuous write.
Similar to write-through caching, but at the whole backup level, not per block.

Alternatively, some type of pre-compression of backups into a large archive, prior to uploading them to the datastore I suspect would work too.

Caveats:
I imagine deduplication would go out the window with this approach, or require a ton of work in future, so would likely need to be disabled for this functionality?

 

[Stefan_R]

As you mention, deduplication does not work well with such an implementation - however, we cannot simply "disable" it, as it's built into the very core of how PBS stores data. Every chunk is an individual file, which can be placed on disk anywhere by the file system it resides on.

I don't think such a thing would be impossible, but certainly hard to implement for generally little reward - then again, PBS is open source, so if you have an idea, the mailing list is open Alternatively, such feature requests are usually better placed on our bugtracker, where they can be discussed more thouroughly.