Hi, Im new to Proxmox VE and PBS so im happy to be corrected if i say anything thats not true.
PVE and PBS dont natively support any kind of immutable backup system. Ive seen you can lock and unlock backups but that seems more to stop accidental deletion rather than preventing a bad actor with access to your systems deleting your backups before he switches on his ransomware attack.
PBS now supports S3 storage datastores which opens up the concept of true immutable backups.
As i understand it the way PBS handles the deduplication and the writing to an S3 bucket using a local cache works really well - in that only new blocks are ever written to the S3 bucket.
Only Garbage collection and Pruning ever request deletion of files and only a Verify or Restore would read significant amounts of data from the store.
Turning on object lock in the S3 bucket effectively gives you immutability with the following caveats.
1. Pruning and Garbage collection generates delete markers for deleted meta file and chunks that are object locked.
2. PBS doesnt currently know anything about object locking and the status of the files as the S3 api just confirms file deletion even on locked chunks.
3. You need to setup lifecycle rules in the s3 bucket to clean up these deleted files and chunks once the object lock has expired.
4. The is an issue where "old" chunks of deduplicated data will have its object lock expire whist it could still be part of an active more recent backup. This leaves it vulnerable to be deleted whilst still needed.
5. chunks and meta files deleted by garbage collection would need to be "restored" in the S3 bucket by removing the delete markers for all files needed for a specific backup using some sort of script in S3
Point 4 above breaks the immutability of the backups. as if you had say 90 days of object lock and say a year later had a bad actor deleting your backups. the object locking for chunks uploaded more than 90 days ago would have had their object lock expire and as such could be deleted which would leave large numbers of missing chunks and destroy a backup.
Point 5 means that to recover an immutable backup from S3 that had been maliciously deleted would require first "undeleting" all the object locked content in the S3 bucket then the data re-imported into PBS
Wouldn't it be great if these 2 features could be built into PBS
1. object lock renewing/extending for chunks still in use by more recent backups (perhaps this would be part of garbage collection)
2. the ability to undelete all chunks associated with a specific backup.
I assume both of these actions can be performed via the S3 api ?
anyway i thought id bring this up for discussion as True Immutable and Deduplicated backups seem to be a holy grail that only Veeam seems to be offering at the moment.
PVE and PBS dont natively support any kind of immutable backup system. Ive seen you can lock and unlock backups but that seems more to stop accidental deletion rather than preventing a bad actor with access to your systems deleting your backups before he switches on his ransomware attack.
PBS now supports S3 storage datastores which opens up the concept of true immutable backups.
As i understand it the way PBS handles the deduplication and the writing to an S3 bucket using a local cache works really well - in that only new blocks are ever written to the S3 bucket.
Only Garbage collection and Pruning ever request deletion of files and only a Verify or Restore would read significant amounts of data from the store.
Turning on object lock in the S3 bucket effectively gives you immutability with the following caveats.
1. Pruning and Garbage collection generates delete markers for deleted meta file and chunks that are object locked.
2. PBS doesnt currently know anything about object locking and the status of the files as the S3 api just confirms file deletion even on locked chunks.
3. You need to setup lifecycle rules in the s3 bucket to clean up these deleted files and chunks once the object lock has expired.
4. The is an issue where "old" chunks of deduplicated data will have its object lock expire whist it could still be part of an active more recent backup. This leaves it vulnerable to be deleted whilst still needed.
5. chunks and meta files deleted by garbage collection would need to be "restored" in the S3 bucket by removing the delete markers for all files needed for a specific backup using some sort of script in S3
Point 4 above breaks the immutability of the backups. as if you had say 90 days of object lock and say a year later had a bad actor deleting your backups. the object locking for chunks uploaded more than 90 days ago would have had their object lock expire and as such could be deleted which would leave large numbers of missing chunks and destroy a backup.
Point 5 means that to recover an immutable backup from S3 that had been maliciously deleted would require first "undeleting" all the object locked content in the S3 bucket then the data re-imported into PBS
Wouldn't it be great if these 2 features could be built into PBS
1. object lock renewing/extending for chunks still in use by more recent backups (perhaps this would be part of garbage collection)
2. the ability to undelete all chunks associated with a specific backup.
I assume both of these actions can be performed via the S3 api ?
anyway i thought id bring this up for discussion as True Immutable and Deduplicated backups seem to be a holy grail that only Veeam seems to be offering at the moment.
Last edited:
