Why hw raid without write back is so much slower than simple sata?

[mmenaz]

This questions bugs me since long time. I know pveperf is not a "full flagghed" benchmark tool, but we know that can give useful and "real" indication of the storage speed.
So this is the question: if I setup a single sata storage, I get FSYNCS/SECOND around 800.
If then I setup a 4xsata RAID10 with hardware controller but no write through, I get something like 160 (that is 5 time slower)!
Don't think that "normal sata" disk has writhe back cache enabled (wouldn't be too risky?), so I would expect that raid performances be at least in pair with single sata, but should be much better in any case (512MB of cache for reads, 4 simultaneus disk to read from).
Second question: I'm testing adaptec 5805z with ZWW and works good, but really so far (I'm new to RAID and BBU) all RAID controller had the bbu problem that is "out of work" in one or two years? Or are LSI Raid (i.e. Logic LSI MegaSAS 9260) that have such a short BBU life? Seem so bad and expensive have to monitor bbu state (and often with proprietary software! I hate it) and have to replace it so often (stop the server, replace the bbu, see checkdisk applied because have been a long time since last check... in short hours of server downtime).
Thanks a lot

 

[dietmar]

Don't think that "normal sata" disk has writhe back cache enabled (wouldn't be too risky?),

Normal SATA disk has write cache enabled by default (and yes, that is dangerous).

Most RAID controller disables the cache on the disk (or you need to do that).

Second question: I'm testing adaptec 5805z with ZWW and works good, but really so far (I'm new to RAID and BBU) all RAID controller had the bbu problem that is "out of work" in one or two years?

What bbu problem do you talk about exactly (AFAIK that controller does not have a battery)?

 

[mmenaz]

What bbu problem do you talk about exactly (AFAIK that controller does not have a battery)?

I mean the problem that BBU last only one or two years, then you have to replace them. I've found a IBM document that says average time is just one year, and it starts from battery manufacture date, not first usage. In other places I've found that lifetime is 2 years. My experience with LSI controller is that, if you don't update controller BIOS, it lasts only few months, and with updated bios in any case should be 2 years. Is it a normal situation? My computer cmos clock battery, not rechargable, for instances lasts 5-10 years! Also if server is KO, bbu can preserve cache data for only 48 hours, so scaring.
Or am I missing something?

 

[dietmar]

Or am I missing something?

adaptec 5805z use SSD to store date (no battery)?

 

[tom]

...
Or am I missing something?

yes, you did not get the "zero maintenance", see http://www.adaptec.com/en-us/products/controllers/hardware/sas/performance/sas-5805z/

LSI calls it CacheVault, e.g. http://www.lsi.com/products/storagecomponents/Pages/MegaRAIDSAS9285CV-8e.aspx

I run several 5805Z, no issues and great performance. here with a raid10, 4 x 2 TB WD RE4 SATA drives (512 mb controller cache is set to "write-back" as its protected by zero maintenance module, WD disk write cache is disabled on all disks as this cache is not protected from powerloss):

pveperf from the root partition (ext3):

pveperf

CPU BOGOMIPS:      19150.93
REGEX/SECOND:      828617
HD SIZE:           49.22 GB (/dev/mapper/pve-root)
BUFFERED READS:    339.64 MB/sec
AVERAGE SEEK TIME: 6.68 ms
FSYNCS/SECOND:     3431.78
DNS EXT:           15.75 ms
DNS INT:           0.62 ms

pveperf from a ext4 partition:

pveperf /space/


CPU BOGOMIPS:      19150.93
REGEX/SECOND:      864369
HD SIZE:           2608.42 GB (/dev/mapper/pve-space)
BUFFERED READS:    285.50 MB/sec
AVERAGE SEEK TIME: 10.80 ms
FSYNCS/SECOND:     3145.56
DNS EXT:           16.24 ms
DNS INT:           0.56 ms

I assume if you run better SAS disk you will get quite amazing speed.

 

[tom]

thomaskrenn published good wiki articles (in German)

general info:
http://www.thomas-krenn.com/de/wiki/Cache_Einstellungen_von_RAID_Controllern_und_Festplatten

http://www.thomas-krenn.com/de/wiki/Adaptec_Zero_Maintenance_Cache_Protection_(ZMCP)
http://www.thomas-krenn.com/de/wiki/LSI_CacheVault_Flash_Cache

 

[mmenaz]

yes, you did not get the "zero maintenance", see http://www.adaptec.com/en-us/products/controllers/hardware/sas/performance/sas-5805z/

Maybe I've no expressed myself well.
I do have lsi with BBU (battery), I do have 5805Z with zero maintenance cache installed (thanks to this forum I knew about it).
My concern is, how is it possible that most are selling controllers with BBU (Battery) if my experience with lsi and BBU has been so bad? If I read around that is "normal" that they last only one or 2 years? If is "normal" that in any case of loss of power on the server, they retain cache data for about 48 hours?

 

[tom]

its even worse. most sell/buy hardware raid controller but they do not buy the BBU which means cache is not working (or on risk). I have seen a lot of servers in many places working like this. this was/is common on most server bundles form the big sellers, I do not name them here but this change only slowly. they also trying to sell fake raid as hardware raid.

at the end the customer got issues and then most blame the software for the fault and loss of data, corrupted systems. windows servers are always on top here but most times its not the fault of windows, its just a wrong hardware setup.

bbu got a very limited lifetime so its quite nice that there is now an alternative.

 

[mmenaz]

its even worse. most sell/buy hardware raid controller but they do not buy the BBU which means cache is not working (or on risk). I have seen a lot of servers in many places working like this.

That is exactly what puzzled me. BBU (or equivalent) is sold as an "option", like "is not needed, buy just if you want to improve the situation".
But as far as I understand, is NEEDED.
If I buy a raid (without BBU) I want:
- more safe data, but if I enable write back I risk 512MB of not saved data, a disaster!
- more feature, but if I don't enable write back, it works 5 times slower!
I thought it could not be true... but you confirm that it is.

bbu got a very limited lifetime so its quite nice that there is now an alternative.

so on top of this crazy situation, I have one more component to monitor and a lot of money / working hours wasted.

Thanks a lot for the clarification.
Wondering NAS/SAN like QNAP TS-879 Pro use (seems an interesting product with 2 GB or 10GB lan interfaces, maybe ideal to use as shared storage among 2 proxmox servers. Wondering if you can contact them and add a "Proxmox" label near the "VMware, Citrix and Hyper-V ready" ).
I really think that proxmox has big potentials in a lot of regards that I can't even imaging or grasp (too costly hardware to experiment with, and too high level knowledge to have acquired as an hobbyist).