[SOLVED] TrueNAS Scale NFS Datastore very slow

[EpicLPer]

Heya,

I'm currently using a setup where I've mounted a NFS store to /mnt and using that as a Datastore inside PBS. However I noticed the chunk generation was really slow... it took almost 30 minutes just to finish. I've also set up the same on my Synology and it was done in around 1-2 minutes there.

Now I'm wondering, what settings could be wrong here? Could it be something directly on TrueNAS that's slowing things down or my fstab mount parameters?
These are the ones I'm currently using: XXXXXXXX.XXX:/mnt/pool_4x12/Backups/Proxmox_BackupServer /mnt/TrueNAS_PBSShare nfs vers=3,nouser,atime,auto,retrans=2,rw,dev,exec 0 0
And the TrueNAS Scale NFS share settings (I had to put "root" in Maproot/Mapgroup so PBS wouldn't throw a "Operation not permitted" error):

 

[EpicLPer]

Hm, seems like it was just chunk generation that was super slow? Backups seem to run at an acceptable speed it seems, not exactly sure.

 

[EpicLPer]

No, seems to run a lot faster on my Synology. So the slow NFS speed on TrueNAS Scale is still an issue

 

[EpicLPer]

I think I figured out a solution for this. I cross-checked the mount options one of my PVE hosts had which had very good speeds and simply used those on PBS too, now the speeds are way above 240MBit/s which is way acceptable

If you want to replicate this simply mount your NFS share on a PVE host first, then run the following in a shell:
cat /proc/mounts

Search for the host you added and copy those parameters over to your PBS host's /etc/fstab, in my case this was (censored some data):
XXXXXXXX.XXX:/mnt/pool_4x12/Backups/Proxmox_Backups /mnt/pve/TrueNAS1_ProxmoxBackup nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=XX.XX.XX.XX,mountvers=3,mountport=44431,mountproto=udp,local_lock=none,addr=XX.XX.XX.XX 0 0

This seems to be the solution for my case at least, your results may vary of course!

 

[Dunuin]

Using "relatime" instead of "atime" could make a big difference as then not every read will cause an additional write.

 

[EpicLPer]

Using "relatime" instead of "atime" could make a big difference as then not every read will cause an additional write.

Yeah, that's potentially what the fix was that I posted boils down to Not a Linux expert or anything and just dabbling along but this definitely seems like it did a trick!

 

[ayo91]

Thanks for your help. Other topic that helped me to get a lot of better perfomance was to set sync=disable in my Truenas dataset. Its a good point if you are using Truenas.

 

[UdoB]

was to set sync=disable in my Truenas dataset.

Yes, it increases the IOPS drastically.

Just keep in mind how this works: the system will signal "okay, your data is written" while it is not - it is lying. The data is just copied into Ram. If you can live with data loss if "something" happens (like a power fail or any kind of Kernel-crash) then this is fine - we are talking PBS here. For precious data (like the documents you are currently working on, hopefully served on another share) is is probably not.

Just my 2 €¢...