NFS Datastore creation fails at 99% (time safety check)

[sergioposadam]

Hello everyone,

I'm having an issue while trying to create a datastore in PBS using NFS. The process fails at the final step after 99% of the chunks are created. However, when I create a datastore on a local drive, such as /tmp/DStest, it works perfectly fine.

I already upgraded PBS to the latest version

Error Log:

...
Chunkstore create: 95%
Chunkstore create: 96%
Chunkstore create: 97%
Chunkstore create: 98%
Chunkstore create: 99%
Chunk metadata was not correctly updated during access time safety check:
Timestamps before update: accessed Some(SystemTime { tv_sec: 1746199830, tv_nsec: 792995000 }), modified Some(SystemTime { tv_sec: 1746199830, tv_nsec: 792995000 }), created None
Timestamps after update: accessed Some(SystemTime { tv_sec: 1746199830, tv_nsec: 792995000 }), modified Some(SystemTime { tv_sec: 1746199830, tv_nsec: 792995000 }), created None
TASK ERROR: access time safety check failed: access time safety check using newly inserted chunk failed, aborting GC!
Error: task failed (status access time safety check failed: access time safety check using newly inserted chunk failed, aborting GC!)

Has anyone else experienced this issue or have any suggestions on how to resolve it? Any help would be greatly appreciated!

Thank you in advance!

 

[cl1705]

I am having the same exact issue.

I have tested this on 2 different NFS shares we will call share A and share B.

Share A is in the same data center in the same relative network as the PBS server
Share B is on a different subnet in a different location that has to go through a firewall

I can successfully mount both NFSs and I was able to create the datastore on share A no issues
Share B however does mount and I can copy data to it, I can finish the chunking, but then fails with that same error.

The only difference I can see is that the latency to share B is two times that of latency to A.

We have tried to adjust the NFS mount parameters with things like lazytime, but to no avail.

 

[waltar]

access time safety check failed

Did you mount your XYZ filesystem inside your NAS with "noatime" and/or "nodiratime" (which then is exposed to your PBS with nfs) ?

Edit: Or did you mount your nfs share on PBS with "noatime" and/or "nodiratime" ?

 

[Chris]

Hi, this check was introduced to exactly for this reason: to early on detect issues with filesystems not honoring access timestamp updates, which are essential for the garbage collection to work correctly (see https://pbs.proxmox.com/docs/maintenance.html#gc-background). Please do share the mount parameters you are using for mounting the NAS as well as details about the filesystem and appliance used for the NAS.