single unencrypted file in encrypted repo - where does it belong to ?

[RolandK]

i have made some weird observation, i cannot explain to myself

i switched my repos to encrypted a while ago and all unencrypted backups have been purged and removed by garbage collection.

in webui all backup snapshots show up encrypted.

for being sure that there is everything encrypted, i checked the repo if really everything is encrypted on disk ( i don't trust gui ).

for that i used tools "entropy" tool from https://github.com/merces/entropy and file utility ( https://bugzilla.proxmox.com/show_bug.cgi?id=7151 )

now, the weird thing is, i found ONE unencrypted file in my repo

# file -m /root/pbs.magic /backuppool/ds-pve2-bnkw-repli/.chunks/bb9f/bb9f8df61474d25e71fa00722318cd387396ca1736605e1248821cc0de3d3af8
/backuppool/ds-pve2-bnkw-repli/.chunks/bb9f/bb9f8df61474d25e71fa00722318cd387396ca1736605e1248821cc0de3d3af8: Proxmox Backup Server unencrypted compressed blob

# entropy /backuppool/ds-pve2-bnkw-repli/.chunks/bb9f/bb9f8df61474d25e71fa00722318cd387396ca1736605e1248821cc0de3d3af8
2.55 /backuppool/ds-pve2-bnkw-repli/.chunks/bb9f/bb9f8df61474d25e71fa00722318cd387396ca1736605e1248821cc0de3d3af8

as we can see this is not encrypted, as file utility is telling that and entropy tool shows value <<8

but where does that file belong to ?

# find . -name "*.fidx" |while read file;do proxmox-backup-debug inspect file $file;done|grep bb9f8df61474d25e71fa00722318cd387396ca1736605e1248821cc0de3d3af8

did not show up anything, so apparently it's not being referenced by the index files

BUT - if i start a GC, then the timestamp of this file is getting updated.

so, maybe this file be referenced by client.log.blob , index.json.blob or qemu-server.conf.blob ?

if yes, why is it unencrypted ?

i checked another backup repository, and it's the same there - there is ONE file with entropy <<8 which appears unencrypted, but it isn't referenced by any of the .fidx inside the same backup repo, at least proxmox-backup-debug inspect file ...fidx won't show it....

file -m /root/pbs.magic.mgc /backuppool/ds-pve-bigiron-repli/.chunks/bb9f/bb9f8df61474d25e71fa00722318cd387396ca1736605e1248821cc0de3d3af8
./.chunks/bb9f/bb9f8df61474d25e71fa00722318cd387396ca1736605e1248821cc0de3d3af8: Proxmox Backup Server unencrypted compressed blob

 

[Chris]

This chunk is the all zero unencrypted but compressed chunk. We use it for the atime update safety check on garbage collection, it being inserted if not present. You might disable the atime safety check (in the datastore tuning options) if you are sure your storage does them as required by garbage collection and do not want this one to be inserted, although no harm in having it present either.

The all zero chunk was choosen for this since it is most likely present in case of an unencrypted VM backup with zero data blocks.

 

[RolandK]

ah, ok, that makes sense. thanks for explaining.

i removed the atime safety check mark and on the next GC the file got removed.

 

[fabian]

so, maybe this file be referenced by client.log.blob , index.json.blob or qemu-server.conf.blob ?

to shed some further light into the internals of PBS: blobs are never referencing chunks. blobs and chunks are the same thing format-wise, just used for different purposes:
- chunk: blob stored in the chunk store, to be referenced by indices via its digest
- blob: blob stored in a metadata dir, for directly storing smaller bits of data (config files, logs, the manifest) associated with a backup snapshot with splitting it up into chunks