Local and S3 backup - almost double space?

[Str1atum]

Hi there!

I configured for testing purposes an additional S3 backup of my VMs/LXCs, but due to the local caching the local backups + S3 cache seem to use almost double the space than before. Is that expected behaviour or is there some workaround?

Best,
Str1atum

 

[Chris]

Hi,
the local cache will grow to use as much space as available on the filesystem it is located on. That is why it is recommended to setup a dedicated disk, partition or zfs dataset with quota for it, see https://pbs.proxmox.com/docs/storage.html#datastores-with-s3-backend

If you did not set that up initially, you could either move or recreate the datastore with a different cache location.

 

[Str1atum]

Thanks, now using a zfs dataset with 128 GB but upload speed of my first backup is much slower than before - how does cache size affect backup speed?

 

[Chris]

The cache will only keep up to it's capacity entries in the local store cache, evicting chunks afterwards. This should however not affect your upload speed until you reached the cache capacity. What operations did you perform to move the cache location? Also, how much slower are we talking about?

Further, note that there are further optimizations such as dirty bitmap at play, which will greatly influence the number of chunks to be uploaded and therefore backup speed.