I need advices for my storage

[proxnano]

Hi,

At the moment my VM are store in server HDD 146Gb SAS in RAID 5 and it's pretty good. I've test with a Netgear ReadyNAS RN104 with 2x 2Tb RAID 0 but the performance are awful.

I hesitate to change my 146Gb SAS to 240GB SSD like this one : http://ark.intel.com/products/94203/Intel-SSD-540s-Series-240GB-2_5in-SATA-6Gbs-16nm-TLC
Do you think that i'll have a real performance boost ?

Otherwise what NAS would you recommend me ?

Thanks a lot

 

[LnxBil]

How many 146GB disks are you using? The more disks, the better, therefore a RAID-0 (should never, never, NEVER be used and) is on 2 disks very slow. You benchmarked 146 SAS 10k or 15k against 7.2k SATA? Yes, size does not always matter.

If you use 1 GBE, you will not saturate the SSDs in your NAS, but you'll be faster than spinning disks - faster with respect to IOPS, not throughput in general.

 

[proxnano]

At the moment i use 4x 146Gb SAS 10k and here is my pveperf :

CPU BOGOMIPS: 22664.92
REGEX/SECOND: 958646
HD SIZE: 94.37 GB (/dev/dm-0)
BUFFERED READS: 206.57 MB/sec
AVERAGE SEEK TIME: 5.95 ms
FSYNCS/SECOND: 2887.50
DNS EXT: 56.64 ms
DNS INT: 57.02 ms (local)
​

I hesitate to add 4 more HDD 146Gb SAS 10k, to change all disks to 8 SSD or to stop store VM direct on my server and migrate to a NAS

 

[LnxBil]

Your numbers look like BBU hardware raid (so that's good), but your CPU is very slow. How old is your CPU? Your DNS also looks very slow (at least to what I'm used to compare that to).

8x 146 GB work fine. If you do not plan to have more than 1 server, install local SSDs. NAS is more complex and is not necessarily faster (yet it costs more, you need more network infrastructure, e.g. 10 GBE etc.). Depending on your RAID controller, you do not need to buy 8 SSDs, because you've already reached bus saturation. They are also horrendously expensive compared to your 146 GB SAS disks.

To be on the absolute safe side, you need to buy SAS SSDs, depending on your controller. I'm running a (ZFS-L2ARC) SATA-SSD for ZFS on my server on a ordinary HP RAID P410 card in a caddy for a defect 146 GB 2,5'' SAS. So, check that in advance.

 

[proxnano]

Thanks, you're right, my CPU is a bit old its a E5440 2Ghz Here is my server https://h10057.www1.hp.com/ecomcat/hpcatalog/specs/provisioner/05/458563-421.htm

My RAID controller is a HP Smart Array P400 : http://h18000.www1.hp.com/products/quickspecs/archives_Division/12400_div_v7/12400_div.HTML

SSD SAS are to expensive for the use of my server but i don't know how to determine the bus saturation of my RAID controller. I don't want to buy 8 SSD SATA if i've reach saturation with only 4.

 

[proxnano]

After a rapid search here is the answer :

"Do not try to use older controllers like P800 or P400 - they are simply too slow and the whole new P410 cost like one SSD drive - you probably installing SSD to get more speed. 2-3 SSD drives will saturate P400 or P800 controller, and they only run SATA at 1.5Gbps - that is too slow. You need a controller that supports SATA at 3Gbps (SAS at 6Gbps)."

Very interresting links : https://community.hpe.com/t5/ProLia...D-drive-instead-of-HP-SAS-drives/td-p/1128019

 

[hansm]

It all depends on your workload. Response time of SSD is really good, from my pveperf:

AVERAGE SEEK TIME: 0.29 ms

You need to determine the storage demand:
- performance
- disk space
- reliability
- costs

Reliability can be the chosen RAID configuration, RAID0 is no RAID! RAID5 can be interesting by means of disk space but it's reliability is doubtful, you better choose RAID6 or even better RAID10. RAID10 comes at the cost of loosing 50% of storage space. Reliability can also be seen as in endurance of SSD, you're mentioned Intel 540S is een consumer grade SSD and can handle 40GB of disk writes per day for 5 years (=71TBW or 0,17 DWPD). It can be used for small setups. I have no experience with these drives, we use Intel S3500/S3510 and S3610 disks and these are great!

A possibility for you is to just buy 2 Intel 540S SSD's (240GB or larger) and put them in RAID1, or ZFS mirror. The ZFS mirror can be extended later by addind 2 additional drives and add them to the ZFS pool, then you have RAID10 and benefit from extra performance and capacity. Maybe you'll saturate the P400 but who cares if you don't need the extra speed of newer controllers? You really benefit from the low response times, high IOPS and high read and write speeds. In PVE you can move VM disk images from your SAS pool to the new SSD pool.

It's just an example. If your speeds are OK now, just add SAS disks for expansion but I don't think they are much cheaper in terms of cost per GB than the 540S 240GB+ SSD's.

 

[LnxBil]

After a rapid search here is the answer :

"Do not try to use older controllers like P800 or P400 - they are simply too slow and the whole new P410 cost like one SSD drive - you probably installing SSD to get more speed. 2-3 SSD drives will saturate P400 or P800 controller, and they only run SATA at 1.5Gbps - that is too slow. You need a controller that supports SATA at 3Gbps (SAS at 6Gbps)."

Very interresting links : https://community.hpe.com/t5/ProLia...D-drive-instead-of-HP-SAS-drives/td-p/1128019

Yes, I feared that. I've tested P400 with SATA also and yes it's limited to 1,5 GBit so you're saturated with a bunch of disks already with the maximum throughput. They are also not capable of running direct 1:1 throughput mode for ZFS, so you're better off with a used SAS2008-based controller with IT-firmware and attach the SSDs directly to that and build a ZFS raid if you want to use ZFS.

Otherwise, I'd suggest to buy a controller which is capable of delivering your SSD performance. One downside is of course that your system is old and you're most certainly limited by the PCIe 2.0 bus.

 

[proxnano]

Thank you all, for your advices. Since my server is really old i think that for the moment i'll only add few 146Gb SAS 10K to upgrade my storage.

But what do you think about using a NAS for the storage ? Could you recommend me a model ?

 

[LnxBil]

I don't use NAS, therefore I cannot give any advice on a model, one some general advice:

You will be slower than you are now if you don't upgrade to 10 GBE and buy a NAS with 10 GBE and at least the number of SAS disks or two/four SSDs and also the network infrastructure accordingly. This'll cost thousands of euro/dollars.