PBS datastore on ZFS dataset without atime

[VictorSTS]

Hello,

We have inherited a PBS server which is storing it's backups in a few datastores. All the datastores are in the same ZFS dataset. Dataset is a raidz1 with around 31TB of storage on HDD. No special device, SLOG or L2arc. atime is off for the pool and the dataset. Everything seems to "work" ok, from backups to prunes, GC and sync tasks. I've tried to restore a few snapshots of different VMs and the VM boots and their filesystems are good.

Given that GC requires "atime" to be enabled so it won't remove unused chuncks with less than 24h and 5m:

- What is GC doing in this case? Does it require more work to do it's job (i.e. higher I/O, longer processing time)? The GC really works, for example:

2022-10-10T04:01:24+02:00: Removed garbage: 224.615 GiB
2022-10-10T04:01:24+02:00: Removed chunks: 190918
2022-10-10T04:01:24+02:00: Original data usage: 53.928 TiB
2022-10-10T04:01:24+02:00: On-Disk usage: 1.89 TiB (3.50%)
2022-10-10T04:01:24+02:00: On-Disk chunks: 1374488
2022-10-10T04:01:24+02:00: Deduplication factor: 28.54
2022-10-10T04:01:24+02:00: Average chunk size: 1.442 MiB
2022-10-10T04:01:24+02:00: TASK OK

- Shouldn't PBS show a warning and ask to enable atime/relatime if this is required? Or isn't it really required?

Thanks!

 

[ukro]

+1 bump

 

[VictorSTS]

Bumping this with lot of tests (please correct me if I'm wrong!) .

I've been testing this with nearly 8 different PBS servers, with datastores on ext4 and ZFS, both with atime, relatime and noatime in the mount options of ext4 or ZFS property. In all cases, Garbage Collector runs flawlessly, removing unused chunks when appropiate. Also, verification passes ok in all cases.

As I understand it, PBS GC does not depend on filesystem options but on filesystem support for atime/relatime. The GC mark phase explicitly changes atime of every chunk [1].

Would really like to know the official Proxmox staff opinion on this and that the documentation mentioned this explicitly.

[1] https://git.proxmox.com/?p=proxmox-...564f7e029dc48304b3aa1293a0f815c9;hb=HEAD#l179

 

[Kodey]

I agree. This is not made clear enough.
I'd like to set noatime on a ssd for endurance/performance purposes but I'm unsure how that will affect pbs.
Also, I'm a bit confused if changing atime causes pbs to consider the file modified and create a new copy.
For example, slocate reads the names of all files. Must new backups be made because all the atimes have changed?

 

[VictorSTS]

I'd like to set noatime on a ssd for endurance/performance purposes but I'm unsure how that will affect pbs.

Will that SSD be used for PBS datastore or is it the source data?

Also, I'm a bit confused if changing atime causes pbs to consider the file modified and create a new copy.
For example, slocate reads the names of all files. Must new backups be made because all the atimes have changed?

I would say no, it won't backup the file again.

For VMs: IRR atime/mtime are stored as metadata in the inode part of the filesystem, at least on ext4. An atime/mtime change in a file will modify a few bits in the disk and given that a VM backup reads the dirty bits since the last backup, only the metadata part of the drive will be dirty and backed up (in blocks of 4MiB). The file itself, it's data, is still the same, there's no need to back it up. Still, if the atime/mtime is stored with the file, only the first 4MiB [1] will be in the dirty map, not the whole file if bigger than 4MiB.

For LXC/file backups: I would have to check the inner workings of the Buzhash algorithm used [2], but I suspect that it will be smart enough to place checksum boundaries in a way that PBS is able to reuse the most chunks.

[1] https://pbs.proxmox.com/docs/technical-overview.html#fixed-sized-chunks
[2] https://pbs.proxmox.com/docs/technical-overview.html#dynamically-sized-chunks