Sign in to follow this  
Bogotron

(Yet another) Do my disk speeds look okay? (3ware raid)

Recommended Posts

Hi.

Longtime lurker registering since I've actually got a reason to post now. :)

Quick system summary: Old Intel BX motherboard, 512MB memory, P3-700 (100MHz fsb), winxp pro, 3ware 7810 with 2xWD1200AB, 2x 120GB Maxtor D540X and 4x 120GB Maxtor DM+9 (3 of them brand new) in raid-5, highpoint rocket133 with a lonely wd2000BB and a 3com nic.

ATTO scores:

First as basic disk:

basic.png

and then as dynamic disk:

dynamic.png

So, going from dynamic disk helped a bit on my ATTO scores. Didn't impove my realworld scores at all though. Read speeds are great, but the problem is sloooow write speeds. I know the 7810 isn't optimized for raid-5 performance, but surely more then 4-5MB/sec should be feasable when copying from the other controller or the network. Although I am running a mix of disks, some 5400rpm and some 7200rpm, would that impact performance this much?

Reading from the raid to either network mounts or the other controller yields great speeds (maxes out the other disk or the nic), but anything writing to the raid is extremely slow. Using perfmon to monitor transfer speeds I get an average of ~4.5MB/sec when copying to the array. Interestingly enough, doing two or three copies to the array at the same time increases speeds by almost 2MB/sec per thread (increasing outstanding I/Os thereby increasing total throughput?). When running three copying threads I get ~8MB/sec and my aging P3-700 is pushing 90% or so, so the CPU is really struggling.

I've tried running w2k, since some people on this board have had better performance running that. For me though the difference was absolutely zero. Another thing I've noticed is that when "pulling" files to the raid volume over the network I get a steady stram of data, while "pushing" data I get a rapid peak<->nothing oscillation (maxtor sawtoothing? at those speeds? and no promise controller in sight?). This occurs in both win2k and winxp.

Latest firmware and drivers for the 3ware are installed, and write-caching is enabled. No extra processes are running in the background and only one process reads or writes to the disk at a time.

So, my question is: How to increase my write performance?

Things I've though of so far:

1. Replace the motherboard + cpu with something more up-to-date. A 1.8GHz P4 based thingy, or an AMD-760 MP motherboard (for a nice 64-bit pci bus) is an option.

2. Replace the WD disks, since people don't really hold them in high regard when it comes to raid performance. Maybe they're holding the array back in some way.

3. Upgrade the 3ware card to a 7500-8.

4. Live with it. I'm mostly going to be reading from the array anyways. Still annoying though.....

Any hints/tips would be greatly appreciated.

Share this post


Link to post
Share on other sites

Hmm... After reading the storagereview.com article comparing the 7410 and the 7450 (7500 series predecessor), some numbers look a lot more meaningful to my kind of use. Down at the bottom of the page the graph showing 4-disk raid-5 performance is fairly... unsettling.

Seems like my work pattern is soley linked to sequential write with 1 outstanding I/O. Actually, if I assume that one explorer process copying files generates 1 outstanding I/O, my measured performance matches the graph extremely well. At least as well as can be expected by correlating my measurements with two sample points on the graph. :)

In short, I should have gotten the 7500-8..... Any of the gurus think such an upgrade sounds like a good idea?

Share this post


Link to post
Share on other sites

Well, terms like raid-5 and single drive don't really mix. :D I'm fairly sure my disk speed problems would go away if I ran something other than raid-5, but raid-5 is unfortunately a necessity.

Share this post


Link to post
Share on other sites

I think I remember somebody having some slow write performance with a 3ware controller mostly fixed by a processor upgrade. Do a search and you should find it.

Share this post


Link to post
Share on other sites

Chew:

Yeah, I remember reading that as well. That was on a 7500 series card (new version with much improved raid-5 performance), and he had speeds around 30MB/sec (iirc) and thought they were too low, which they are compared to the advertised 40-65MB/sec. I think he was running on a 1.4GHz P4 celeron or something simmilar, and upgraded to a 1.8GHz Northwood.

Anyway, I've returned the 7810 in exchange for a 7500-8. When I get it (Monday? Tuesday?) I'll run some more benchmarks and post my new (and hopefully improved) scores. From what I've gathered, the 7810 is simply very slow at sequential raid-5 writes.

Share this post


Link to post
Share on other sites

You've got 1MB and 2MB cache drives (and I can't remember what the DM+9's are - too many new Maxtor models), and you're mixing 5400rpm and 7200rpm. Having such a mixture of disks may not be helping the situation either. If you're still not satisfied with the performance once you get the new controller try creating a RAID 5 with just the DM+9 drives to see how that performs.

Share this post


Link to post
Share on other sites

These where the number I was getting with my 7810 running in a 32bit PCI with 8 IBM 80G (IC35L080ABER07) in RAID 5 and firmware/driver 7.4

ATTO on RAID 5

When I upgraded to 7.4 I gained some performance, haven't retested it since I went the latest update.

Want to see slow? Try 3 Maxtor 60G 540's in RAID 5 on a 6410 8O I was lucky to get 4M-7M writes even as JBOD it took over 2 hours to ghost 10G work of data.

I'd bench each drive on its own, Simple Raid 5 is only as big as the smallest drive and as fast as the slowest drive.

Share this post


Link to post
Share on other sites

Got the new controller today, and first thing I did was to run some benchmarks. :)

ATTO on the new controller:

7500-8.png

Since the interesting part is the write speed you can all see that the peak speed has actually dropped off a bit, whereas the minimum speed (the one I saw with real life use) has increased considerably. Real life use has also shown a corresponding improvement in speed.

The current speed is okay (15-20MB/sec write speeds are good enough for use as a NAS box), but in the longer term I'm probably going to upgrade the motherboard+cpu, which should (hopefully) yield another 1.5-2x speed increase, even on my hodgepodge mixture of disks. And by that time I guess drive sizes (and my data needs) will have increased to a point where I need to add another array anyway.... :D

Unfortunately, I wasn't able to create a raid-5 volume on the Maxtor DM9+ disks and bench them alone to determine the impact of my odd mix of disks and spindle speeds, mainly due to the fact I have to vacate all my data from the server I used for backup as soon as possible.

Share this post


Link to post
Share on other sites

Forgot to mention why I think a cpu+mb upgrade would improve my speeds: When running ATTO, my cpu usage is 100% with 85-95% spent in kernel time. I'm most probably cpu bound.

Share this post


Link to post
Share on other sites
Forgot to mention why I think a cpu+mb upgrade would improve my speeds: When running ATTO, my cpu usage is 100% with 85-95% spent in kernel time. I'm most probably cpu bound.

Fwiw, here are my Atto scores with the same settings as yours:

attotest012903.gif

The high read score seems somewhat wacky to me but I don't know enough to say whether anything shown can be trusted (confirmation from anyone would be appreciated). And, IIRC, there's another thread here that says Direct I/O should be off but that gives totally screwy results.

Btw, it also saturated the CPU here when running.

Relevant specs are:

7500-4 running JB1200s in RAID 1, .036 Firmware, .044 Bios, dynamic disk on (Win2KSP3)

2100 Tbred-B @2.25GHz, FSB=180MHz (A7N8X, PCI locked)

1GB Corsair 3200

Hth.

Share this post


Link to post
Share on other sites

Just wanted to update this thread just in case anyone is interested. I've replaced the two WD disks in the array with two maxtors (DM9+).

maxtor%20only.png

Read speeds have increased almost 20MB/sec at large transfer sizes. Next up: New motherboard and CPU since I'm still running at 100% CPU when accessing the disk.

sleon:

And, IIRC, there's another thread here that says Direct I/O should be off but that gives totally screwy results.

Tried turning it off and got some exceptionally high scores. I want my disks to be fast, but 250MB/sec read and write speeds on a 33MHz/32-bit PCI bus is a little too much IMO. :)

Share this post


Link to post
Share on other sites

seee here: 3ware + 7.6 driver

The Spec is also very outdated, but have a good write speed. That is not my card, I just take some benchmark when help my client to install it, so I don't know how does this card with old driver will look like

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this