shared storage VM read/write speeds slow with TrueNAS

peregost

New Member
Jan 25, 2022
3
0
1
44
Hi everyone,

I am new to the Proxmox environment and most of my setup is for learning, experimental purposes, and testing, but I am seeking guidance as to why I am unable to resolve my slow speeds when benchmarking a VM's mounted network drive that is using NFS shared storage.

I get around 110mb/s read | 15 mb/s write on a VM instance with 10gb shared storage via TrueNAS nvme raid1 pools. Obviously if the VM is using a nodes local storage, the read write speeds are as one would expect with raid1 ssd's (500mb/s both ways).

A quick run down of my cluster/network and storage share setup:

3 Super Micro 1U's
24 core x2 cpus
128gigs ram
2ssd's raidz1 for local/os
2ssds raidz1 for zfs pool
10gb nic

TrueNAS server:
MSI gaming motherboard
i5
16GB ram
10gb nic
4 1tb ssds
2 1tb nvme's
no caches setup

Ubiquti Aggregation Switch
all 10gb om3 fiber cables
utilizing only 1 10gb port per nic on each node including TrueNAS server.

The 2nvme's installed in my TrueNAS server are what I am mounting the VM shared storage to in Proxmox, they're configured in a raid1 pool using Iz4 compression and NFS share.

Also wanted to mention, if the VM is restarted (configured with windows 10 os) it takes an extremely long time to boot into windows, around 15-20 minutes.
I do not see where a bottle neck issue would be coming from, but hoping someone on here can provide information on what could possibly be the issue.
 
Last edited:
Hi,

I have a similar setup and i'm also having strange read/write speeds.

1682347559101.png

This is one example, sometimes its the write that is really slow and the read is good.

I'm using ZFS over iSCSI though.


Edit:
I disabled sync in the pool options:
1682349332860.png

And now i have correct read/write speeds.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!