how do PBS calculate a-time

J

jwsl224

Member
Apr 6, 2024
72
3
8
in the past, pbs would refrain from deleting chunks for 24 hours "for file systems that have a low a-time".

we are using zfs file system. i know zfs has an a-time for all blocks. does pbs use that? or something of its own? if it's not using zfs a-time, i wanna turn it off for the performance gain.

i'm asking about this so i can safely set the access time cut-off for garbage collection without causing data loss. cause 24 hours 5 minutes is a little too long; we turn over too much data for that.
 
the 24h (which is still the default) was used as a precaution because we were worried about "weird" filesystem implementations that would use lazytime/relatime like semantics for explicit timestamps updates like our GC uses to mark chunks. we haven't actually seen any reports of those in the wild. we did have a few reports about storage appliances completely ignoring explicit timestamp updates while pretending they were supported and done, so recently additional checks were implemented to ensure atime updating seems to behave, and the option to reduce the threshold to allow faster cleanup of garbage was introduced at the same time.

with ZFS reducing the atime threshold should be safe irrespective of atime settings on the ZFS dataset - PBS doesn't rely on the atime being updated on regular file access (which is what relatime/atime control), but just that explicitly updating the atime timestamp via `utimensat` works.
 
  • Like
Reactions: Johannes S
fabian said:
with ZFS reducing the atime threshold should be safe irrespective of atime settings on the ZFS dataset - PBS doesn't rely on the atime being updated on regular file access (which is what relatime/atime control), but just that explicitly updating the atime timestamp via `utimensat` works.
Click to expand...
ok, so, just to make sure i fully understand here, pbs is using utimensat, which is a linux feature, to change the "modified date" timestamp on the chunk file "with nanosecond precision" (according to the docs), and the zfs atime and relatime are not being used at all and can be turned off?
 
almost - we use utimensat to change the atime timestamp, but that should (and does for all filesystems we've encountered so far) work irrespective of the atime related mount options - those only concern automatic atime updates when the file is accessed using regular means.
 
  • Like
Reactions: jwsl224
Morning :)- just on this same issue - and just to make sure I'm not missing anything:

@fabian - are you saying that previous occasional reports of chunks being 'mistakenly' deleted is not at all related to zfs atime being on or off (and can thus be safely turned off) - and we can turn zfs atime for the PBS storage pool off?

In a 200TB-usable PBS storage pool with 12x 3-way mirrored vdevs in a ZFS pool running HDDs (I know - we should ideally be using SSDs, but for backup economics), with a 3 way mirrored SSD special vdev and a 3 way mirrored SSD SLOG, apart from moving the storage pool to SSD to improve performance, turning atime off safely is a very serious option to improving pool performance (in addition to adding more vdevs and possibly adding additional separate stand-alone storage pools).

Your view? Would atime off and relatime on be the way to go?
 
Last edited:
I don't think atime=off will give you a lot of performance benefits, in particular if you already had relatime=on (the default). with ZFS, you need atime=on and relatime=on to get relatime behaviour, setting atime=off will turn of automatic atime update altogether.
 
We run a verification after every single backup event (of which we have more than 400 backup events/24 hours, backing up the equivalent of approx 60TB of actual data/24 hours, dedup'ed down to about 10%) - atime would almost certainly have an impact :)

From this previous post (https://forum.proxmox.com/threads/zpool-atime-turned-off-effect-on-garbage-collection.76590/) it seems that atime NEEDS to be on.

I just want to be 100% sure of the proper setup - there have been various posts which seem to allude to the fact that atime being off is ok.

What's the official Proxmox position - once and for all? So I can tick that off my list :)
 
Last edited:
like I said in my initial reply - we were initially worried about some storages/filesystems implementing timestamp updates weirdly, and trying to extend atime/relatime/lazytime behaviour to those as well. we haven't actually found a single instance of that. ZFS we explicitly tested - it makes no difference to PBS GC whether atime is on or off (either fully, or as relatime). this is also why we implemented the "atime" check at the start of GC - it checks that touching a chunk has the desired effect, and should catch ill-behaved storages. we did actually see those a few times - some network storage appliances that had an option to skip timestamp updates altogether, while telling the caller that the update went okay..
 
  • Like
Reactions: Johannes S, Angelo and Chris
  • Like
Reactions: Johannes S, Angelo and fabian
You must log in or register to reply here.
Top Bottom