KCComp

The Final Word on 'SCSI performance in Windows XP'

Recommended Posts

Here is the last, the final, definitive scoop on XP SCSI performance.

It's really time for us to close the thread 'Terrible SCSI Performance in Windows XP". 678 posts, and no one reads any of the previous information because there's too much of it. This leads to a kind of self-perpetuating rant, which serves no purpose other than to keep circulating a myth about SCSI and XP.

So here's the bottom line, followed by a FAQ.

Windows 2000 had a minor bug where application requests to write through the cache (i.e. ignore the drive/controller cache and write directly to the disc) were not honored. In other words, writes were being cached even if the application asked for them not to be cached. This provided artificially high write performance in Windows 2000 for such applications (which are rare). Unfortunately, a number of benchmarks (e.g. ATTO) incorrectly used the WRITE_THROUGH flag in an attempt to provide what they thought was the most accurate, uncached, performance result.

Windows XP correctly honors the WRITE_THROUGH flag for Basic disks. When people upgraded from Windows 2000 to Windows XP, and ran a benchmark like ATTO, they saw greatly "decreased" write scores. That's because the writes were no longer being cached. While this is an accurate measurement of totally uncached writes, it's not the kind of performance result that ATTO wanted to provide their users.

Many people found that changing from Basic to Dynamic disks in Windows XP "reclaimed" their performance in ATTO. This is because Microsoft forgot to fix the WRITE_THROUGH issue with the Dynamic disk code path. Dynamic disks do not "reclaim" any "lost" performance, because the performance was never lost. It's an artificial benchmark phenomenon. Oh, sure, some file copy operations were affected, because they were using the WRITE_THROUGH flag. That's by design to ensure files are copied/moved with integrity. It doesn't affect 99.9% of system performance on non-server platforms.

The next version of Windows will likely have a checkbox under the "enable write caching" checkbox in the disk properties, to turn on and off the ignore WRITE_THROUGH flag behavior (for people who are using battery-backup and/or UPS and are therefore sure of disk integrity). In the meantime, users can choose one behavior or the other by choosing Basic or Dynamic disks.

OK, so that's the bottom line, please read over it carefully. Here is the FAQ.

Q: Is there a Windows XP BUG that reduces SCSI performance?

A: Absolutely NOT.

Q: Should I move back to Windows 2000, or to some other OS, due to this issue?

A: No, read the opening statements of this post again.

Q: Is this a SCSI related issue? Does it affect IDE or IDE RAID?

A: This is in no way SCSI related. The only reason it occurs on SCSI-class devices (SCSI & IDE RAID) and not on standard IDE, is that IDE does not have a WRITE_THROUGH flag in its command set. This could be considered a flaw in the IDE specification, which I understand is due to be rectified in a future spec revision. Note that any IDE controller which has its own drivers is a SCSI-class device and will appear as such (e.g. Promise Ultra100) even if it isn't RAID.

Q: If this is just due to ATTO writes being uncached, why is my write performance SO SLOW? I should have xxMBps to the platter...

A: The only way you can truly have sequential writes is if they are cached in the drive buffer. Without a cache, you can only write a chunk at a time, and in between chunks, you'll have missed the heads coming around and have to wait on rotational latency. That's why you'll see writes peaking after a certain block sized is reached in ATTO.

Q: You said I could choose behaviors by using Basic or Dynamic disks. What if I have XP Home, which doesn't support Dynamic disks?

A: SCSI (and IDE RAID) are high-end / enthusiast platforms. It would be very unusual for anyone to run XP Home on such platforms. For example, workstation-class machines from Dell, HP, etc. are not even offered with XP Home.

Q: How do I convert to Dynamic disks?

A: Control Panel, Administrative Tools, Computer Management, Disk Management, right-click on "Disk n" and choose "Convert to Dynamic Disk". Read the online help about Dynamic Disks to make sure you understand the restrictions, etc.

Q: Does XP Service Pack 1 affect the issue?

A: No.

Q: I heard that Windows XP was "made for the masses" and therefore "isn't optimized for best performance". Are there any tweaks I can use to improve disk performance?

A: Not really. That whole "made for the masses" stuff sounds slick but in fact, XP is very self-optimizing. Defragment your disk often, delete temp files and temporary internet files first. Be aware of an IE bug that leaves extraneous files in your Temporary Internet FilesContent.IE5 directory under randomly generated temporary directory names (basically, it dumps a file there every time you hit Stop while a page is loading, or navigate to another page before a page completely loads). Establish a paging file on each physical drive. Ensure write caching is enabled (some SCSI controllers will cause the write caching flag to be turned off on every reboot unless you set the controller to turn on write caching in its BIOS). Avoid converting FAT32 to NTFS, format as NTFS in the first place.

Q: I think you're wrong about this, because I have xxx SCSI performance problem under Windows XP.

A: There are dozens of reasons you may be experiencing poor performance that have nothing to do with this issue, and are not due to any global issue with SCSI performance under Windows XP.

KC

Share this post


Link to post
Share on other sites

#1 - starting other threads to "summarize" the main one just creates fragmentation of the discussion. Please discuss in that thread for f/u, not here.

Having said that, in case someone just reads this...

#2 - several of us affected by the bug have demonstrated real world drops in large transfers of 50% or more under XP. This is not due to the write through flag or any benchmark error. Try copying 4+ gb of data under w2k and then XP and seeing that it takes twice as long under XP (3 physical drives - copying from one data drive to another, both separate from boot drive). Such performance hits are real world (and quite apparent even without the stopwatch) and given the only thing different was (clean) installs of both operating systems, it's hard to come up with any other explanation.

FWIW

Share this post


Link to post
Share on other sites

Posting it in the original thread would just get lost in the noise. I posted a link to this post in the main thread.

In regard to your copy issue, read my post again, I addressed that.

KC

Share this post


Link to post
Share on other sites

Not sure that I can take your word unless you can provide where you got this information. I do agree with the 700+ thread being full of too much BS that isn't necessary.

Share this post


Link to post
Share on other sites
I wish I could believe what you're saying but plz explain why some are getting great(normal) readings in ATTO while others are not ?

The drivers for some controllers apparently have their own way of dealing with the write-through flag. I happen to own both sorts, one that suffers and one that doesn't.

Leo

Share this post


Link to post
Share on other sites

Interesting theory. And it seems to make sense. All we need now is a utility for WinXP to force writethrough cache on or force it off. Do you know if simply explorer file copies enable it or not? I would think to achieve fastest performance you want it enabled.

Share this post


Link to post
Share on other sites
Interesting theory.  And it seems to make sense.  All we need now is a utility for WinXP to force writethrough cache on or force it off.  Do you know if simply explorer file copies enable it or not?  I would think to achieve fastest performance you want it enabled.

No, definitely not! Microsoft used to advise developers for Win9x to use the write-through flag "for performance reasons." So they are to blame that it is used in some applications. In reality, the flag reduces performance. Unless you fear loss of unsaved data in case of a crash, you want to have it disabled.

Leo

Share this post


Link to post
Share on other sites

yes, Can you explain 'real' word performance differences??

My stopwatch doesnt lie. Try copying a 1GB file from any source to the SCSI drive under XP and do the same under 2K.

I'm not discrediting you, but I don't think you know what the 'real' issue is.

Share this post


Link to post
Share on other sites

KCComp,

I can't follow your reasoning.

1. You claim that a bug in W2K resulted in "artificially high performance" for certain type of apps.

If indeed so, why does the Explorer in W2K copy 2 GB files twices as fast as the Explorer in XP?

Either Explorer is buggy (requests wrong write cache behaviour) or then your "artificially high peformance" argument is bogus.

Which way is it?

Either the SCSI code is broken or the way XP copies through Explorer is broken. NOTE! I'm not talking about any benchmark utility now. Just plain old File Explorer copying with LARGE SEQUANTIAL writes exceeding cache sizes by manifold.

2. You claim "Is there a Windows XP BUG that reduces SCSI performance?"

So if there is no SCSI bug, then why does the Explorer copy take twice as long on XP as it does on W2k?

Is there a bug in the Explorer, is this what you mean?

If so, then there is a bug in XP (that affect scsi performance) as well and again your reasoning is false.

3. You argument that "There are dozens of reasons you may be experiencing poor performance that have nothing to do with this issue".

Are you implying that EVERYBODY in the previous thread on this same issue is a technical moron and that they can't configure their computers properly?

That they have a SCSI performance problem that is a result of "OTHER" problems, rather than what is discussed in arguments 1 and 2?

All in all, I don't think your arguments stand as a whole.

If you want to express them so that us, technically inadept morons also understand them, then please feel free to clarify :)

Regards,

Halcyon

Share this post


Link to post
Share on other sites

I had the same clearly demonstrable problem using xp vs. 2k - copy 1-2 gb of large files and just by using a simple stopwatch the difference is easily measured and significant (I was seeing more than twice the performance under 2k). This is not benchmarking, it's real world use. In fact that's how I first came to know about this problem - copying gigabytes of files from one disk to another for a backup and realized how damn long it was taking me to do it under XP.

Just another datapoint, I can't add anything else.

Share this post


Link to post
Share on other sites
1. You claim that a bug in W2K resulted in "artificially high performance" for certain type of apps.

If indeed so, why does the Explorer in W2K copy 2 GB files twices as fast as the Explorer in XP?

Either Explorer is buggy (requests wrong write cache behaviour) or then your "artificially high peformance" argument is bogus.

Which way is it?

The former.

No one is trying to claim that it's only a "benchmark issue." It does manifest itself in real world usage, but NOT ALL applications are affected. Only those that indiscriminately use a certain flag.

I suspect file copy uses that flag by default, simply because it has far greater accuracy requirements under any possibl conditions.

Leo

Share this post


Link to post
Share on other sites
I suspect file copy uses that flag by default, simply because it has far greater accuracy requirements under any possibl conditions.

Or because the file that's being copied simply might not fit in the cache. :)

Leo

Share this post


Link to post
Share on other sites

While we're on the subject, can someone explain to me why my favorite big program takes 15 seconds to load in XP, versus 10 seconds in Win2K? This is off the exact same Cheetah X15-36LP, I even did separate clean installs of both OSes, w/updated Adaptec drivers in XP, same result.

Tyan Thunder K7 mobo w/onboard Adaptec 160 SCSI

Cheetah X15-36LP 36.7GB

Share this post


Link to post
Share on other sites
While we're on the subject, can someone explain to me why my favorite big program takes 15 seconds to load in XP, versus 10 seconds in Win2K?  This is off the exact same Cheetah X15-36LP, I even did separate clean installs of both OSes, w/updated Adaptec drivers in XP, same result.

And what program might that be? By the way, it might have nothing to do with hard drive performance.

Leo

Share this post


Link to post
Share on other sites

Geez guys. KCComp's explanation was clear and comprehensive. He addressed many of the questions that are being raised in confusion quite adequately in his post... at least, it seems that way to me.

Share this post


Link to post
Share on other sites
Geez guys. KCComp's explanation was clear and comprehensive. He addressed many of the questions that are being raised in confusion quite adequately in his post... at least, it seems that way to me.

Yes, he did a very good job.

Leo

Share this post


Link to post
Share on other sites
Geez guys. KCComp's explanation was clear and comprehensive. He addressed many of the questions that are being raised in confusion quite adequately in his post... at least, it seems that way to me.

The effort is appreciated, but everyone is questioning the validity of his conclusions.

Share this post


Link to post
Share on other sites
The effort is appreciated, but everyone is questioning the validity of his conclusions.

Well, it's not like he's guessing anything. He's just summarizing what has long since been corraborated by many sources, including from Microsoft's own word.

Leo

Share this post


Link to post
Share on other sites

The effort is appreciated, but everyone is questioning the validity of his conclusions.

Well, it's not like he's guessing anything. He's just summarizing what has long since been corraborated by many sources, including from Microsoft's own word.

Leo

I'll admit I haven't been following the topic too much lately, but did MS ever confirm that the file copy function in Windows/Explorer uses the flag?

And has anybody seen them mention that the Dynamic Disk code doesn't have the issue fixed? There seems to be plenty of evidence that changing to Dynamic Disks hasn't incorrectly improved performance for all users in the huge XP SCSI thread. I know it's possible many are seeing completely seperate performance issues, but I don't think that applies in all cases.

While I agree largely with KC's summary, I still think it doesn't fit 100%..

Share this post


Link to post
Share on other sites

I have seen all of KCComp's assertions before -- some from MSFT directly, and some from other forum members who know Windows architecture well. cas, for example, can tell you all about how the Explorer copy function works... what API's and flags it uses... in fact, he improved on it and gave a few of us this optimized version.

Share this post


Link to post
Share on other sites

No matter if we are getting incorrect ATTO scores in win2k etc, but if the real world file copy performance in winXP with SCSI disks is slow compared to win2k, then there is a problem, and it hasn't been solved yet.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now