SAN/NAS Benchmark - Good speeds?

[hotwired007]

Hi Guys,

I've spent today running benchmarks to find out whether my NAS (ReadyNAS 2100 4x 1TB) is running as fast as i need and whether iSCSI or NFS is the fastest method - I have already decided that i'm going to have to use iSCSI so that my backups work correctly but i was interested in getting some results to compare. I've used the newest version of CrystalDisk (3.0) on a number of drives:

Windows 2003 - Proxmox 1.9 - (NFS) - C:\ (IDE)
50MB	1000MB
Read (MB/s)	Write (MB/s)	Read (MB/s)	Write (MB/s)
71.55	40.43	64.52	35.01
71.86	40.49	34.59	25.65
6.325	6.065	1.249	1.265
6.738	6.632	1.496	1.475

Windows 2008 - Proxmox 1.9 - (iSCSI) C:\ (IDE)
50MB	1000MB
Read (MB/s)	Write (MB/s)	Read (MB/s)	Write (MB/s)
66.33	50.77	58.29	46.56
66.61	51.55	31.55	49.47
6.806	6.804	1.767	6.655
6.991	6.931	1.808	6.974

1.5TB SEAGATE SATA 3
50MB	1000MB
Read (MB/s)	Write (MB/s)	Read (MB/s)	Write (MB/s)
111.6	89.71	100.2	96.12
43.32	78.42	27.83	45.25
0.668	0.928	0.313	0.62
1.427	0.842	0.565	0.558

Dell Mirrored RAID Dual 1TB Drives
50MB	1000MB
Read (MB/s)	Write (MB/s)	Read (MB/s)	Write (MB/s)
82.66	76.71	101.7	101.4
42.22	81.75	27.81	45.1
0.783	0.577	0.344	0.69
3.076	0.535	1.619	0.637

USB 2.0 - 320GB WESTERN DIGITAL
50MB	1000MB
Read (MB/s)	Write (MB/s)	Read (MB/s)	Write (MB/s)
27.66	26.23	29.11	27.82
18.51	27.74	17.62	23.96
0.378	1.387	0.331	0.752
0.539	1.409	0.449	0.791

Although on the NAS box i have slower sequential read/writes than an individual local drive, it is faster than anything else on the other benchmark tests.

Currently its running only 1x 1Gb network port, would having the second enabled help to improve this?

Can anyone else validate this? has anyone got anything else to compare against?

 

[marotori]

Your limit is cased by the network layer.

In my experience... 1gb links are not fast enough.

Local disk: 300gbs
1gb link: 90gbs if you are lucky!

So.... Bonding helps. 2 cards will give you 150gps or close to that.

If you want speed.... Fibre channel is really the only answer


Sent from my iPhone using Tapatalk

 

[udo]

Y...
Local disk: 300gbs
1gb link: 90gbs if you are lucky!
...

????
1gb can give not 90 times speed?! I assume you mean 90MByte/s over an 1GB-Connection.
Local disk (Sata II) use 3gbs (serial), which is theoretical 300 MByte/s.

Udo

 

[marotori]

Sorry... That is exactly what I meant.

I should read what I write more carefully before submitting it!


Sent from my iPhone using Tapatalk

 

[hotwired007]

Local disk: 300gbs
1gb link: 90gbs if you are lucky!

So.... Bonding helps. 2 cards will give you 150gps or close to that.

I've bonded the two ports on the SAN and on the proxmox box.

From online calculations the MAXIMUM should be 250MB/s.

I'm getting 64.25MB/s read and 30.97MB/s write

What other tweaks can i do to increase performance?

I'm looking a building a database server on this system but with that kind of IO it'd end up getting bogged down...

 

[marotori]

I've bonded the two ports on the SAN and on the proxmox box.

From online calculations the MAXIMUM should be 250MB/s.

I'm getting 64.25MB/s read and 30.97MB/s write

What other tweaks can i do to increase performance?

I'm looking a building a database server on this system but with that kind of IO it'd end up getting bogged down...

Are you using NFS? or ISCSI?

And how have you made that caluclation of speed?

The only system I know that gets near wire speed is good ol' ftp. And even that would not be 250MB/s

Remember.. complex protocols. (SMB,SSH,NFS,ISCSI,etc) have lots of hand shaking. End result is you loose bandwith just by the need to use them.

I did say this a few posts back.. but dont expect wonders.

With NFS, you can turn off imediate writing to disk. This means that the client can be freed up while letting the nfs do its work in the background. My experience says this is HIGHLY risky and liable to loose data if you crash the system mid write.

There are also numerous tweaks to the TCP stack that can help.. but I dont think you will ever get the speed you want without Fibre Channel.

A year or two back I purchased a massic 16tb raid system. Set the whole thing up with nfs etc...

2 months on I ditched it because the performance of the IO sensitive apps was so rubbish. (nas had 4 bonded 1gb links) (client had 2gb links bonded).

I have now gone back to local storage. My next move will be local - with drbd sync. This will be a good option without doing the full enterprise san

Rob

 

[hotwired007]

I was hoping to get atleast 90MB/s with the bonding, there doesnt seem to be any performance increase at all.

The connnections are NFS currently, although i'm going to change to iSCSI.

 

[marotori]

What type of bonding have you used?

 

[hotwired007]

Adaptive load balancing

 

[marotori]

I suggest...


802.3ad

And jumbo frames.

Should be a bit better

 

[hotwired007]

Jumbo frames are enabled...

 

[kbrault]

Rob, your comment: "My next move will be local - with drbd sync. This will be a good option without doing the full enterprise san" ..... would that not put you back to the network being the limiting factor in performance?

 

[hotwired007]

found out that my switch (netgear GS724T) doesnt support the full 802.3ad so have left it at ALB, everything seems to run fine aslong as the servers aren't doing loads of IO, have decided that high IO servers will run from local disks rather than network storage.