[SOLVED] TrueNAS Scale NFS Datastore very slow

E

EpicLPer

Member
Sep 7, 2022
47
8
8
30
Austria
epiclper.com
Heya,

I'm currently using a setup where I've mounted a NFS store to /mnt and using that as a Datastore inside PBS. However I noticed the chunk generation was really slow... it took almost 30 minutes just to finish. I've also set up the same on my Synology and it was done in around 1-2 minutes there.

Now I'm wondering, what settings could be wrong here? Could it be something directly on TrueNAS that's slowing things down or my fstab mount parameters?
These are the ones I'm currently using: XXXXXXXX.XXX:/mnt/pool_4x12/Backups/Proxmox_BackupServer /mnt/TrueNAS_PBSShare nfs vers=3,nouser,atime,auto,retrans=2,rw,dev,exec 0 0
And the TrueNAS Scale NFS share settings (I had to put "root" in Maproot/Mapgroup so PBS wouldn't throw a "Operation not permitted" error):

1711619554638.png 1711619570853.png
 
I think I figured out a solution for this. I cross-checked the mount options one of my PVE hosts had which had very good speeds and simply used those on PBS too, now the speeds are way above 240MBit/s which is way acceptable :)

If you want to replicate this simply mount your NFS share on a PVE host first, then run the following in a shell:
cat /proc/mounts

Search for the host you added and copy those parameters over to your PBS host's /etc/fstab, in my case this was (censored some data):
XXXXXXXX.XXX:/mnt/pool_4x12/Backups/Proxmox_Backups /mnt/pve/TrueNAS1_ProxmoxBackup nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=XX.XX.XX.XX,mountvers=3,mountport=44431,mountproto=udp,local_lock=none,addr=XX.XX.XX.XX 0 0

This seems to be the solution for my case at least, your results may vary of course!
 
Using "relatime" instead of "atime" could make a big difference as then not every read will cause an additional write.
 
Dunuin said:
Using "relatime" instead of "atime" could make a big difference as then not every read will cause an additional write.
Click to expand...
Yeah, that's potentially what the fix was that I posted boils down to :) Not a Linux expert or anything and just dabbling along but this definitely seems like it did a trick!
 
Thanks for your help. Other topic that helped me to get a lot of better perfomance was to set sync=disable in my Truenas dataset. Its a good point if you are using Truenas.
 
ayo91 said:
was to set sync=disable in my Truenas dataset.
Click to expand...

Yes, it increases the IOPS drastically.

Just keep in mind how this works: the system will signal "okay, your data is written" while it is not - it is lying. The data is just copied into Ram. If you can live with data loss if "something" happens (like a power fail or any kind of Kernel-crash) then this is fine - we are talking PBS here. For precious data (like the documents you are currently working on, hopefully served on another share) is is probably not.

Just my 2 €¢... :-)
 
  • Like
Reactions: Johannes S
Thanks all! This was hugely helpful. I have been running a Synology for years, but am slowly moving to my new Truenas Machine. I couldn't for the life of me figure out why my iperf3 numbers looked great, but my write speeds to an nfs share on truenas was horrible compared to my Syno NAS....especially since both were running 10G connections.
 
Well TrueNAS also allows to run VMs or LXCs with it's newest release so I would expect way better performance if you setup PBS inside a TrueNAS VM or a LXC. For LXC there is a writeup in this forum: https://forum.proxmox.com/threads/pbs-on-truenas-have-your-cake-and-eat-it-too.162860/ In the same thread community members also wrote obout their experience with VMs hosted on TrueNAS and you will find more on this with the search function for the forum. Hosting PBS directly on the TrueNAS should result in way faster backup/restore/garbage collection and verify jobs since the storage then is storage without the latency issues of a network share. If you absolutely need to use a network share (due to resource constraints on your NAS device etc) you should go with ISCSI for better performance. It's covered by following writeup: https://jrehkemper.de/content/linux/proxmox/truenas-iscsi-storage-for-proxmox-backup-server/
 
  • Like
Reactions: UdoB
You must log in or register to reply here.
Top Bottom