KCComp

The Final Word on 'SCSI performance in Windows XP'

Recommended Posts

That's too bad.  If KCComp (and many others) are correct, the results should be similar between operating systems, and we can put this issue to rest.

Any takers?

I still had about a factor of 10 performance difference in real world copies between BSD and 2k/XP. On my new array it's down to about a factor of 3 difference, but then BSD was pretty much maxing out PCI on my old array.

I'd find it less irritating if there were utilities that didn't bog down at such low speeds so I could use them in place of explorer and such.

Share this post


Link to post
Share on other sites

hold on KC, your whole argument is based on 2K vs. xp. with this whole "artificially high write performance in Windows 2000" issue, you haven't taken into account the users still running 9x. i know for a fact that scsi preformance in 9x is similar to 2K.

does 9x have the same "artificially high write performance"?

i've been researching the topic on another forum, and i was pointed here.

just for the record, i've read through the first 300 posts in the original thread. it's bookmarked and i'll be reading the rest tomorrow. if, when i'm done, i see that nobody has pointed out some of the fixes i've found around the net, i'll post them here.

Share this post


Link to post
Share on other sites

Have any of you benched a RAID array with Winbench 99 lately?

I was having TERRIBLE performance on pairs of WD1000JBs on an Adaptec 2400A, as measured by ATTO, and HDTach (of course) as a low-level benchmark was completely off the mark. Then I ran some good old Winbench 99s and...

SingleWD1000JBonIDEWinbench.gif?dc=4675395610449152821

2WD1000JBsWinbenchDIT.gif?dc=4675395610440680601

3WD1000sWinbenchDIT.gif?dc=4675395610443512097

(forgive me for the visual size of the graphs. It was impossible to see the numbers at a smaller size. These are crunched to 20k gifs, reasonable.)

...So it looks like the Adaptec 2400A is working after all; looks like WD/JB drives aren't too bad in RAID 0 after all; and looks like Windows XP's problem with SCSI devices does not extend to RAID cards as badly as it may have appeared.

- OR - perhaps it just looks like Winbench 99 was programmed to use whatever caching was available, rather than attempting to set a bypass flag. Ya think?

... which is why, unlike the low-level benchmarks like HDTach, Winbench 99 might be a reasonable measure of RAID performance in Windows XP. I'm not experienced enough to know for sure. I just know I like these Wb99 graphs better than the very low scores I was seeing on ATTO, HDTach, and IOmeter before.

Share this post


Link to post
Share on other sites
Guest Eugene
That's too bad.  If KCComp (and many others) are correct, the results should be similar between operating systems, and we can put this issue to rest.

Any takers?

KCComp's post deserves this crosspost:

Alright, I bit the bullet and did what I should have done a long time ago when cas was willing to devote his time the SR community by programming his copy utility.

The following tests were run utilizing cascopy.exe in default mode copying a file 674.496,512 bytes in size from drive F: to drive G: partitioned to maximum sizes utilizing NTFS. Figures are an average of three trials which all presented reasonably precise results:

DM+9 200 GB to DM+9 200 GB, WinXP: 46.8 MB/sec

DM+9 200 GB to DM+9 200 GB, Win2k: 44.6 MB/sec

Delta moving to 2k from XP: -4.7%

X15-36LP 36 GB to X15-36LP 36 GB, WinXP: 45.9 MB/sec

X15-36LP 36 GB to X15-36LP 36 GB, Win2k: 47.2 MB/sec

Delta moving to 2k from XP: +2.8%

These figures were drawn utilizing all standard SR Testbed3 controls. Can anyone imagine a more controlled testing environment? :)

Share this post


Link to post
Share on other sites

At last, some real figures!

Thank you Eugene.

I am modifying cascopy right now, so that the buffer size and queue depth can be controlled from the command line. I will also be adding a flag to control FILE_FLAG_WRITE_THROUGH. Those with enough time on their hands should be able to track any issues across service packs and disk type (basic, dynamic).

Although the differences between implementations might be exacerbated by a smaller buffer size, or shorter queue depth, they hardly seem worth reverting to w2k for.

Share this post


Link to post
Share on other sites

I am an idiot for that November 4th post of mine {above}.

Forgive me for being so off topic. All I can say is either (a) I thought I was in another thread when I hit "submit", or (B) I was considering RAID controllers part of the SCSI realm because of the way the drivers work, or © both. Forget it anyway, I sold that crappy Adaptec.

Share this post


Link to post
Share on other sites
Guest Eugene
Although the differences between implementations might be exacerbated by a smaller buffer size, or shorter queue depth, they hardly seem worth reverting to w2k for.

Agreed. The question is whether this massive tide of anti XP-SCSI sentiment that's swept not just SR but the enthusiast community as a whole can be stemmed.

One simply needs count the number of recent threads here depicting many users agonizing over whether they should use 2k instead of XP simply because of ths supposed xp bug.

Regards,

Eugene

Share this post


Link to post
Share on other sites
I wish I could believe what you're saying but plz explain why some are getting great(normal) readings in ATTO while others are not ?

The drivers for some controllers apparently have their own way of dealing with the write-through flag. I happen to own both sorts, one that suffers and one that doesn't.

Leo

Leo can you or anyone else give examples of controllers that are known to still work OK under XP ? Is this also driver version specific? I would think bios/driver version could be critical...

Thanks

Tex

Share this post


Link to post
Share on other sites
Guest Eugene
Leo can you or anyone else give examples of controllers that are known to still work OK under XP ?  Is this also driver version specific? I would think bios/driver version could be critical...

Unless your usage pattern consists predominately of running atto powertools and copying files to SCSI drives, al controllers "Work OK under XP."

Share this post


Link to post
Share on other sites
Leo can you or anyone else give examples of controllers that are known to still work OK under XP ?  Is this also driver version specific? I would think bios/driver version could be critical...

Unless your usage pattern consists predominately of running atto powertools and copying files to SCSI drives, al controllers "Work OK under XP."

Might one not suggest that "copying files to SCSI drives" is going to be pretty important in a reasonably significant number of usage patterns? Just a thought.

Share this post


Link to post
Share on other sites
Leo can you or anyone else give examples of controllers that are known to still work OK under XP ?  Is this also driver version specific? I would think bios/driver version could be critical...

Unless your usage pattern consists predominately of running atto powertools and copying files to SCSI drives, al controllers "Work OK under XP."

I understand.... (long sigh.....) Now back to the original question again..... What controller do you own that still seems to show good XP performance?

Thanks again.

Tex

Share this post


Link to post
Share on other sites
Guest Eugene
Might one not suggest that "copying files to SCSI drives" is going to be pretty important in a reasonably significant number of usage patterns?  Just a thought.

Perhaps, though applications copying and writing files to SCSI drives is not the same thing as copying files to SCSI drives utilizing Windows XP Explorers built-in copy utility.

As a result, my statement should have read like this:

"Unless your usage pattern consists predominately of running atto powertools and copying files to SCSI drives using XP Explorer's built in utility, all controllers 'Work OK under XP.'"

Share this post


Link to post
Share on other sites

Eugene - can I take it from your last comment then you have seen or verified the poor performance using windows XP explorer copy command?

Not to re-hash old news, but this is what brought me to this thread in the first place- I was seeing tremendous performance drop-off when copying large blocks of files under XP. As I do some video editing, and have quite a few MP3 files that I move from drive to drive, often gigs at a time, it is QUITE noticable. I think this affects more real world use that you might think.

So this really might turn out to be just a windows explorer bug? If I used another copy utility it wouldn't affect the performance?

Share this post


Link to post
Share on other sites
Guest Eugene
Eugene - can I take it from your last comment then you have seen or verified the poor performance using windows XP explorer copy command?

Yep... my own experiences are buried in the long thread somewhere. Followups on it got lost as I got mired in other projects though... my apologies. I've been wanting to run a program like cas' for a long time but have never had time time along with the free testbed time until yesterday (and I still really didn't have time then! I'm currently assessing performance differences between 60 GB/platter, 68 GB/platter, and 80 GB/platter DM+9s...).

Not to re-hash old news, but this is what brought me to this thread in the first place- I was seeing tremendous performance drop-off when copying large blocks of files under XP.  As I do some video editing, and have quite a few MP3 files that I move from drive to drive, often gigs at a time, it is QUITE noticable.  I think this affects more real world use that you might think.

So this really might turn out to be just a windows explorer bug?  If I used another copy utility it wouldn't affect the performance?

KCComp's post here really does answer all relevant questions regarding this issue. This artifact is an issue with Win2k's explorer. Total Copy, similarly, either builds on and utilizes Windows explorer's copy function itself or also similarly ignores this flag.

If you use XP, you could use a copy utility such as cas' to copy your mp3s back and forth and speeds similar to Win2k. He doesn't seem confident enough with it for use in production systems, but perhaps you can persuade him to make it more robust. ;)

Share this post


Link to post
Share on other sites
snip

Okay i'm sorry if i'm beating a dead horse...

but what does this mean about other programs? ie. games? and boot up?

is winXP doing things the same/just as fast as win2k?

and are the findings about cascopy and windows explorer true also for RAID?

maybe i'm just looking for that definitive "yes" or "no" answer from Eugene :)

i <3 Eugene :wink:

Share this post


Link to post
Share on other sites
Guest Eugene
Okay i'm sorry if i'm beating a dead horse...

but what does this mean about other programs? ie. games? and boot up?

Games write to SCSI drives at the same speed under XP as they do under 2k. XP's bootup is generally much faster than 2k, unrelated to any mythical SCSI write problems.

is winXP doing things the same/just as fast as win2k?

WinXP writes to SCSI drives just as fast as Win2k does. Whether WinXP is as fast as Win2k as a whole is beyond this discussion, and again not relevant.

and are the findings about cascopy and windows explorer true also for RAID?

RAID has nothing to do with this issue. Again, irrelevant to the issue of SCSI write speeds in XP. Off, topic, I'd bet the majority of folks who use RAID0 are not experienced significantly increased performance for they're usage and may in fact be feeling a detrimental effect.

maybe i'm just looking for that definitive "yes" or "no" answer from Eugene :)

The definitive answer is that the "XP SCSI write bug" as hypothesized in myriads of threads does not exist.

Share this post


Link to post
Share on other sites

Now wait a minute - I think that windows copy commands are something we all use frequently. If performance hit when doing so is more than 50% as I've demonstrated in real-life testing, then clearly it does exist.

It may be a windows XP copy command bug, but nevertheless it causes a major slowdown in the real world when copying multi-gig directories. I'm talking several minutes of waiting around.

I don't see how one can argue that win2k had a bug that made it faster and now MS has "fixed" the bug by cutting write performance in half. If that's the case, I hope they don't try to fix many other bugs!

Share this post


Link to post
Share on other sites
Now wait a minute - I think that windows copy commands are something we all use frequently.  If performance hit when doing so is more than 50% as I've demonstrated in real-life testing, then clearly it does exist.

It may be a windows XP copy command bug, but nevertheless it causes a major slowdown in the real world when copying multi-gig directories.  I'm talking several minutes of waiting around.

I don't see how one can argue that win2k had a bug that made it faster and now MS has "fixed" the bug by cutting write performance in half.  If that's the case, I hope they don't try to fix many other bugs!

Whether the bug is inherent to SCSI or to implementation of the copy utility in Windows explorer, this is significant.

Nobody that I know uses an alternate file manager in Windows. Since Eugene seems so confident about his diagnosis, I would pressume that this Windows Explorer bug is something that Microsoft should be able to reproduce without effort and thus fix. Let us hope that the fix does not break other things if it ever does come.

Why hasn't Microsoft looked into this and produced a conclusive statement about it after all these months?

Given the importance of SCSI to the viability of any server, why didn't it occur to Microsoft to do its own internal testing, produce a utility such as the one cas provided it for us and verify or disprove the claims about XP SCSI performance? If it has done this, why hasn't it told its users? And more importantly, where is the Windows explorer bug-fix?

Share this post


Link to post
Share on other sites
Guest Eugene
Now wait a minute - I think that windows copy commands are something we all use frequently.  If performance hit when doing so is more than 50% as I've demonstrated in real-life testing, then clearly it does exist.

I said the XP-SCSI write bug as hypothesized in all these threads does not exist. What exists is an XP explorer copy routine that's undeniably slower than Windows 2k's original explorer copy routine. Folks took this discrepancy in explorer copy routines and extrapolated this to mean that all write performed by all applications will exhibit the same performance penalty under XP. This is not true.

It may be a windows XP copy command bug, but nevertheless it causes a major slowdown in the real world when copying multi-gig directories.  I'm talking several minutes of waiting around.

As stated by KCComp, the bug is not in the XP explorer copy routine, it is in 2k's.

I don't see how one can argue that win2k had a bug that made it faster and now MS has "fixed" the bug by cutting write performance in half.  If that's the case, I hope they don't try to fix many other bugs!

This ad hominem attack against Microsoft is unnecessary and irrelevant.

Typical users copy files utilizing xp's explorer, yes. Most typical users do not spend a large portion of their computing time copying large files to SCSI drives utilizing xp's file explorer. If you do, then yes, utilizing xp's explorer to copy files will hinder your productivity. As stated before, however, it is possible for someone like cas to program a utility to make this just as efficient for users such as yourself under xp as it is under 2k. Perhaps the guy who programmed TotalCopy could do it too.

What is incorrect and unfair is to imply that the reduced SCSI write performance exhibited by xp explorer when contrasted to 2k's explorer implies that such reduction occurs in other applications. This, again, is the general argument hypothesized in the myriad of XP-SCSI threads and is not true.

Share this post


Link to post
Share on other sites

I spent quite a few hours benching WIN2K, XP Pro, & NT4 with ATTO. Using a stopwatch to compare file transfers between 2 SCSI data disks, XP was a smidge faster than 2K (by about 1-2 %). Unfotunately, I only used Dynamic disks in my stopwatch testing. I have posted a bunch of ATTO comparison graphs here if anyone is interested.

Share this post


Link to post
Share on other sites

Yuyo, the people you're speaking to aren't going to answer because you called it a "bug", which doesn't fit with their appraisal of the situation. They've repeatedly explained that rather than being a bug, it's just that Microsoft decided not to use cached writes in Explorer's copy routine, for safety reasons, so they patched.

But I feel we can still reasonably ask why our SCSI drives feel much slower on nearly ALL tasks in any version of XP compared to 98.

Share this post


Link to post
Share on other sites
Guest Eugene
Nobody that I know uses an alternate file manager in Windows. Since Eugene seems so confident about his diagnosis, I would pressume that this Windows Explorer bug is something that Microsoft should be able to reproduce without effort and thus fix. Let us hope that the fix does not break other things if it ever does come.

Agreed. Folks who use Windows XP are going to use its explorer to copy files. As verified by my own testing, copying files to SCSI drives using XP explorer will result in performance about half as fast as the same operation using win2k's unfixed explorer. This is undeniable and is not under dispute. What I and some others before me dispute is the implication that this routine of copying files using xp/2k explorers is representative of the performance difference that all other applications will exhibit under the same conditions. It is this false connection that drives so many readers to angst when trying to choose between these OSes.

Share this post


Link to post
Share on other sites

When you are copying multi-gigabyte files, you are looking at straight STR, the subsystem cache wouldn't have any real effect there, as it would with multiple small file copy tasks.

And Eugene - how do you explain that changing to dynamic disk changes the results - does this undo the fix of the bug?

I really appreciate your input on this - I don't mean to come off as overly harsh on MS but lets face reality here - there have been write ups of this problem all over the net and nobody from MS released any information to explain what is going on, leaving us to speculate and frustrate for months.

Leave it to SR to have to dig up the answer that MS should have been able to provide on day 1 of this controversy.

Share this post


Link to post
Share on other sites
Guest Eugene
When you are copying multi-gigabyte files, you are looking at straight STR, the subsystem cache wouldn't have any real effect there, as it would with multiple small file copy tasks.

You're mistaking issues... we're not speaking of subsystem caches, as KCComp outlined in the initial post

And Eugene - how do you explain that changing to dynamic disk changes the results - does this undo the fix of the bug?

I mean absolutely no offense when I ask this, but have you carefully read the first post in this thread lately?

Many people found that changing from Basic to Dynamic disks in Windows XP "reclaimed" their performance in ATTO. This is because Microsoft forgot to fix the WRITE_THROUGH issue with the Dynamic disk code path. Dynamic disks do not "reclaim" any "lost" performance, because the performance was never lost. It's an artificial benchmark phenomenon. Oh, sure, some file copy operations were affected, because they were using the WRITE_THROUGH flag. That's by design to ensure files are copied/moved with integrity. It doesn't affect 99.9% of system performance on non-server platforms.
I really appreciate your input on this - I don't mean to come off as overly harsh on MS but lets face reality here - there have been write ups of this problem all over the net and nobody from MS released any information to explain what is going on, leaving us to speculate and frustrate for months.

Leave it to SR to have to dig up the answer that MS should have been able to provide on day 1 of this controversy.

Actually, folks from Microsoft close to the Windows code in question have released information. Unfortunately who they are is not readily visible. Why are they being so coy and not blaring such credentials? I have no idea on that.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now