Slow read/write values on NFS export

[ninin06]

I am testing before migrating from ESXi7.0 u3 to Proxmox VE8.3. Now I have noticed that the performance of the VM disk on Proxmox is half or even less fast. I have a NetApp with 800GB SSDs for storage. Everything is connected over 10G to a Cisco Nexus switch and the host hardware is the same, so I just swapped ESXi with Proxmox.

I have read some reports of poor NFS performance. When will this be improved? Because this is a reason not to switch to Proxmox and look for other alternatives like XCP-ng.

 

[waltar]

Edit your /etc/pve/storage.cfg and add "options" with nconnect like
nfs: netapp
export /vol/...
path /mnt/pve/netapp
server <netapp-IP>
content rootdir,images,iso
nodes pmx1,pmx2,pmx3,pmx4
options nconnect=4
prune-backups keep-all=1

reboot or let your netapp auto-remount by manual "umount -l".
After node is rebooted or share remounted do check and set nfs read :
mount|grep nconnect
m=/mnt/pve/netapp # change to your name
cat /sys/class/bdi/$(mountpoint -d $m)/read_ahead_kb
echo 8192 > /sys/class/bdi/$(mountpoint -d $m)/read_ahead_kb

Test again.

 

[ninin06]

Changing the nconnect option has improved the performance. However, it is still below the ESXi values SEQ1M Q1T1, RND4k Q32T1, RND4K Q1T1 in CrystalDiskMark. The values are approx. 2/3 or 1/2. I have achieved equivalent results with SEQ1M Q8T1.

Changing the read_ahead_kb did not really make a difference.

ESXi

Proxmox:

 

[waltar]

If you are measuring inside a win vm assign an additional disk to it with "virtio block" interface and cache settings "write back" in pve setup window. (Format with ntfs in vm inside and mount.) Then measure on that disk again.

 

[ninin06]

Now with the “write back” option, the first two values have risen by a lot, even above ESXi. I don't quite understand why the read values of 2400MB have increased so much. However, the performance for the last two values is not yet at ESXi level.

 

[waltar]

What's the usage inside your win vm ? If you handle files the random results doesn't any matter at all otherwise if doing database inside do the nfs read_ahead to 4096.

 

[ninin06]

This Win VM is only for testing the performance on the Proxmox environment. But there will be about 40 VMs mainly AlmaLinux with some Windows servers per host running on the NetApp with various Web/DB servers and ActiveDirectory.

 

[waltar]

So for linux and web services do read ahead of 8192 and for win db or ad service do 4096.