New production enviroment and strange Disk speeds

Aron Dijkstra

Well-Known Member
Aug 6, 2016
41
1
48
43
Hello,

Thanks for tanking the time reading this,

We just bought our new Proxmox production enviroment based on our trials with some cheap second hand hardware we bought.

Our production network is equiped with the following hardware:

2x Supermicro 1U servers with Xeon E1240 CPU with 48GB RAM and Samsung SM863 120GB local disks.
1x Intel NUC Celeron with 8GB for Quorum.
1x Synology RS815+ with 3x WD4000FYYZ and one Samsung SM863 240GB SSD for caching (read)
Connected to a HP 1920-24G Switch.

All servers have 2 ports bonding and the NAS 4port bonding (LACP)
For now Jumbo Framing is not used.

The NAS has one Volume RAID 5, btrfs and 2 shares (ISO files and Image store)

At the moment we are running acceptance tests. But we get strange speedtests. The tests we run are the following:

1 Debian 8 server on SATA disk 1GB 32GB 1 CPU SAMBA.
2 Debian 8 server on SATA disk 1GB 32GB 1CPU on NFS.
3 Debian 8 server on VIrtio disk 1GB 32GB 1CPU on SAMBA.

OUr NFS uses 32K packets.

We have created a 1024MB test file and did a copy with pv (pv 1024mb.test > test2.file)
Copy speed on server 1 arround 52MB p/s Server 2 2.44MB p/s server 3 arround 52 MB p/s.

We did a copy on the proxmox console we got with the NFS share about 33MB p/s and with SAmba 111MB p/s.

When copying with NFS i see a very very high Iops on the Synology (between 500+ to 850+ iops)
With Samba the synology is almost at rest.....

Does anyone have maybe an idea?

With pveperf the FSYNC is as follows:

SMB: FSYNCS/SECOND: 597.61
NFS: FSYNCS/SECOND: 59.23
THIS IS THE SAME SHARE! only diffrent protocol!

There are no other running VM's currently running.

Again thanks for reading and i hope you will have a guess or maybe a a direction we can look in.

With the highest regards,

Aron
 
NFS has a lot of settings. Asking google will give a lot of reading, also follow the vendor (qnap) recommendations for NFS clients

And use a reliable benchmarking toolset, so you can compare different settings.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!