KCComp

The Final Word on 'SCSI performance in Windows XP'

Recommended Posts

But come on guys.... We are talking about trying to maintain data integrity on a file copy operation that might fail because of power loss or something now? It still would fail without the the flag right? This would protect only on copys that had seemed to finish but still set in the cache when the machine went down for gods sake? Think about the odds of this really happening.... This isnt the OS cache even but the disk cache or controller cache now...

I think the flag was supposed to bypass the OS cache but was never supposed to bypass the disk cache and thats what has happened. There are MS knowledgebase articles that back up that the flag was not intended to bypass the disk cache. To say the old copy way is unsafe and MS was doing this to protect themselves is nonsense since this was mainly a XP problem for a long time and 99 percent of those box's are sold with IDE disks that copy the old fast way. so 99 percent of the XP users are still at risk.

The flag was never supposed to bypass the disk cache. Only the OS cache and maybe the controller cache on a big caching scsi controller. But we got scsi manufacturers that went to far and bypassed the disk cache itself. Thats why sometimes a certain driver/bios combo works right and another wrong even for the same controller right now. No one knew how bad they had implemented their solution till now. With the way MS has fixed it. The next fix should come from the scsi manufactures to fix their implementaion of the drivers to not skip the disk cache also. No one knew how bad it was now.

Tex

Share this post


Link to post
Share on other sites
I think the flag was supposed to bypass the OS cache but was never supposed to bypass the disk cache...

The only problem with this hypothesis, is that FILE_FLAG_NO_BUFFERING, documented just below FILE_FLAG_WRITE_THROUGH does exactly this.

Share this post


Link to post
Share on other sites
I think the flag was supposed to bypass the OS cache but was never supposed to bypass the disk cache...

The only problem with this hypothesis, is that FILE_FLAG_NO_BUFFERING, documented just below FILE_FLAG_WRITE_THROUGH does exactly this.

Thats why I said the scsi controller guys mucked up. We jsut never had a real world reason to see the effects before. I sent you my atto's that still had the reads mucked up but your patch fixed the writes. Do you think those reads were hitting a disk cache? I don't.

The differant scsi controller makers all seemed to interpret what they were supposed to do differantly.

Tex

Share this post


Link to post
Share on other sites

Ignoring for the moment the importance (or lack thereof) of write cache settings:

Of all the hubbub surrounding this issue… very few individuals appear to be discussing the fact that Microsoft has acknowledged this as a problem and provided a “fix”. But the proposed solution as it is presented in the link below, is itself flawed.

http://support.microsoft.com/default.aspx?...kb;en-us;308219

Purportedly, if one applies XP-SP1, all is wonderful again. The difficulty arises in the fact that the service pack does not update the NTFS.SYS file to the version identified in the Knowledge-Base remedy (9.28.01 v5.1.2600.14). More to the point, the updated NTFS.SYS file is not even INCLUDED in the SP1 distributions. The file version in SP1 is the same old 5.1.2600.1106

I have searched in vain for the “updated” version.

Does no one else find this to be a bit peculiar?!

Share this post


Link to post
Share on other sites
Ignoring for the moment the importance (or lack thereof) of write cache settings:

Of all the hubbub surrounding this issue… very few individuals appear to be discussing the fact that Microsoft has acknowledged this as a problem and provided a “fix”.  But the proposed solution as it is presented in the link below, is itself flawed.

http://support.microsoft.com/default.aspx?...kb;en-us;308219

Purportedly, if one applies XP-SP1, all is wonderful again.  The difficulty arises in the fact that the service pack does not update the NTFS.SYS file to the version identified in the Knowledge-Base remedy (9.28.01 v5.1.2600.14).  More to the point, the updated NTFS.SYS file is not even INCLUDED in the SP1 distributions.  The file version in SP1 is the same old 5.1.2600.1106  

I have searched in vain for the “updated” version.

Does no one else find this to be a bit peculiar?!

I don't know where you got the idea that XP had a fix. It does not from MS anyway. CAS provided us a "filter" of sorts that basicaly blocks the flag from being based I think.

There is no MS fix at this time for XP or win2k after sp3. Your hope is that they implement the changes they have made to .Net Server 2003 (XP server they claim) as I am runnng .Net Server RC2 and they have a fix for it. I have other posts here describing it in more detail.

Tex

Share this post


Link to post
Share on other sites
I don't know where you got the idea that XP had a fix. It does not from MS anyway. CAS provided us a "filter" of sorts that basicaly blocks the flag from being based I think.

In actuality, I am not stating that M$ actually provided a remedy for the issue. Simply that they _say_ they have provided a fix in the above link, and the fix itself isn't valid.

The proposed solution from M$ doesn't appear to revolve around cache flags.

Share this post


Link to post
Share on other sites
I don't know where you got the idea that XP had a fix. It does not from MS anyway. CAS provided us a "filter" of sorts that basicaly blocks the flag from being based I think.

In actuality, I am not stating that M$ actually provided a remedy for the issue. Simply that they _say_ they have provided a fix in the above link, and the fix itself isn't valid.

The proposed solution from M$ doesn't appear to revolve around cache flags.

They were not fixng this problem. Lots of false hoped and rumors from them in the last year. don't hold your breath.

Tex

Share this post


Link to post
Share on other sites
Ignoring for the moment the importance (or lack thereof) of write cache settings:

Of all the hubbub surrounding this issue… very few individuals appear to be discussing the fact that Microsoft has acknowledged this as a problem and provided a “fix”.  But the proposed solution as it is presented in the link below, is itself flawed.

http://support.microsoft.com/default.aspx?...kb;en-us;308219

Purportedly, if one applies XP-SP1, all is wonderful again.  The difficulty arises in the fact that the service pack does not update the NTFS.SYS file to the version identified in the Knowledge-Base remedy (9.28.01 v5.1.2600.14).  More to the point, the updated NTFS.SYS file is not even INCLUDED in the SP1 distributions.  The file version in SP1 is the same old 5.1.2600.1106  

I have searched in vain for the “updated” version.

Does no one else find this to be a bit peculiar?!

The .14 version is newer than the version found originally in XP. This includes a fix for another performance related problem, but not the problem in question here. The version in SP1 is .1106, which is LATER than the .14 version, and includes the fixed code that was first implemented in .14.

Share this post


Link to post
Share on other sites

Interesting that this is still being discussed.

The current Microsoft recommendation to server OEM's is to either

1) Not use IDE

2) Disable write-back cache on all IDE drives, due to data corruption issues.

What KC is saying is absolutely correct. NTFS relies on and issues write-through requests to maintain metadata coherency, by issueing the FUA (Force Unit Access) command. IDE drives not honor FUA (as it does not exist in the ATA spec). FUA requests WILL slow down performance, but give increased reliability.

However, FUA is ONLY enabled in the following circumstances.

SCSI Basic disk, Win2kSP3

SCSI Basic disk, WinXP

SCSI Basic/Dynamic, Win2003

For example, a stream of sequential writes tagged with the ForceUnitAccess bit can reduce throughput to a single request completed per rotation! That is because by the time a request has been reported as successful and the next request is ready to start, the disk has rotated slightly beyond the start of that request and thus an entire rotation must occur between requests. This would explain why certain benchmarks show a significant slowdown..

Of course, as we all know, STR plays a very small part in overall workstation performance...access time is far more important. However, if it means so much to you, switching to dynamic disk will prevent FUA commands from reaching the disk. However, then your data will be at risk..unless...

You have proper power backup. As a rule, any system with write-caching enabled, should ALWAYS be hooked up to a UPS. Unless FUA is enabled, this is the ONLY way you'll be safe. If you're stuck with IDE, the only way to prevent possible corruption is to either disable write cache, or use a UPS.

Share this post


Link to post
Share on other sites

As others have said, this is B.S.

My "stopwatch test" of Copying a large file, and Pasting it to the same drive completes 5 times faster on a WD 1200JB than on my Cheetah X15-36LP, in WinXP.

No such problem in Windows 2000.

The summary at the top of this thread is absolutely not correct.

Share this post


Link to post
Share on other sites
Guest Eugene
As others have said, this is B.S.

My "stopwatch test" of Copying a large file, and Pasting it to the same drive completes 5 times faster on a WD 1200JB than on my Cheetah X15-36LP, in WinXP.

No such problem in Windows 2000.

The summary at the top of this thread is absolutely not correct.

I believe you need to read this thread more carefully and completely... try out cas' filter too and give him some feedback.

Share this post


Link to post
Share on other sites
Geez guys. KCComp's explanation was clear and comprehensive. He addressed many of the questions that are being raised in confusion quite adequately in his post... at least, it seems that way to me.

The effort is appreciated, but everyone is questioning the validity of his conclusions.

You must be kidding. He clarified a lot of things that you wouldn't dream of doing.

Share this post


Link to post
Share on other sites

Yes. My reply was ridiculous in light of the middle of this (much more interesting than I first thought) thread. Sorry for making you reply, Eugene.

My tests with cascopy back up what you saw in your XP vs. 2K test early in the thread. As always, cas has the right of it.

Share this post


Link to post
Share on other sites

every one can say what ever they want. It still doesnt explain why my x15 36LP takes twice as long to write a 1 gig file to itself then my 10K III does in XP

XP "bug" or not.

One thing i do believe. is a lot of peoples problumes with pour performance in XP with SCSI doesnt have to do with XP

But it sure gives for an easy scape goat.

I sure would like to know why my x15 35LP gets trummped by my 10K III. I stopped worring about. I whent back to 2000 and I'm fixing to order a 10K IV soon.

Killl two birds with one stone.

Share this post


Link to post
Share on other sites

Pico, note that "going back to 2000" is NOT going to fix the problem. Win2k SP3 has the same fix as WinXP on this issue. Of course, you could run SP2, but that gives you all kinds of other problems...

Seriously, the quick and easy solution is just to use dynamic disk under XP. This will also disable FUA requests.

Share this post


Link to post
Share on other sites
Interesting that this is still being discussed.

The current Microsoft recommendation to server OEM's is to either

1) Not use IDE

2) Disable write-back cache on all IDE drives, due to data corruption issues.

What KC is saying is absolutely correct. NTFS relies on and issues write-through requests to maintain metadata coherency, by issueing the FUA (Force Unit Access) command. IDE drives not honor FUA (as it does not exist in the ATA spec). FUA requests WILL slow down performance, but give increased reliability.

However, FUA is ONLY enabled in the following circumstances.

SCSI Basic disk, Win2kSP3

SCSI Basic disk, WinXP

SCSI Basic/Dynamic, Win2003

For example, a stream of sequential writes tagged with the ForceUnitAccess bit can reduce throughput to a single request completed per rotation!  That is because by the time a request has been reported as successful and the next request is ready to start, the disk has rotated slightly beyond the start of that request and thus an entire rotation must occur between requests. This would explain why certain benchmarks show a significant slowdown..

Of course, as we all know, STR plays a very small part in overall workstation performance...access time is far more important. However, if it means so much to you, switching to dynamic disk will prevent FUA commands from reaching the disk. However, then your data will be at risk..unless...

You have proper power backup. As a rule, any system with write-caching enabled, should ALWAYS be hooked up to a UPS. Unless FUA is enabled, this is the ONLY way you'll be safe. If you're stuck with IDE, the only way to prevent possible corruption is to either disable write cache, or use a UPS.

Seriously, the quick and easy solution is just to use dynamic disk under XP. This will also disable FUA requests.

What do you think? I have FAT32 with XP on both SCSI & IDE disks. Do OS use FUA command in this case or not?

My SCSI 15K drive write performance sucks with FAT32 also. So my OS don’t need to maintain metadata coherency BUT only MS want that we all go to Dynamic disks with NTFS. Why so? It’s very big difference between tasks of WEB server, Database server and Work station. I want to have at home SCSI based work station with maximum possible disk performance. Only owner of this PC (not MS) ought to decide use or not to use IDE or SCSI, power backup, define percentage of task where he need good STR or better access time, en. or dis. write-back cache, what kind of firewall, antivirus software to use and so on. So I want from MS choice. Why give this choice for IDE case but not for SCSI? Maybe MS looking forward for SATA corporate disk systems I really don’t know. But new SP2 for XP will be closer to the end of this year. I put all disk system (18files) from Windows .NET Server RC1 to my XP pro but received the same picture – SCSI write disk to self performance sucks. Somebody I think Tex wrote about additional option in Window NET Server 2003 RC2 to enable really SCSI write-back cache. At this moment I trying to find RC2 files and substitute all disk I/O system in my XP pro. But if it will give nothing what to do – go back to W98 or wait for GUI improvement of Linux?

Want to add – nothing personal against MS. Company have complete rights to define their strategic policy.

Details of my system, my tests and other comments about this issue in this tread http://forums.storagereview.net/viewtopic....r=asc&highlight

Share this post


Link to post
Share on other sites

Tex is correct, Win2003 will have a "power managed" option that enables full Write-back capability in systems hooked to a UPS.

As far as your poor performance, well..FAT32 is garbage...Switch to NTFS and you'll be happier.

Share this post


Link to post
Share on other sites
Tex is correct, Win2003 will have a "power managed" option that enables full Write-back capability in systems hooked to a UPS.

As far as your poor performance, well..FAT32 is garbage...Switch to NTFS and you'll be happier.

I explained why I leave NTFS partition and go for FAT32 in above mentioned thread – to no need to bother about NTFS files indexes, files meta data, journaling files system. I only want to see correct implementation of write back cache for all application and OS copy machine for both FAT32 and NTFS. Can only add that for the home (non corporate) user with SCSI or IDE system disk less then 40Mb FAT32 partition will be quicker and much more manageable. As for me NTFS separately have nothing in itself to decide above mentioned problem – very low disk to self copying SCSI throughput on NT core OS. It may be because of absence of the integrity of all transactions via disk cache, adapter cache and system cache forward and back (writes and reads) in the case of system disk-to-self copying for all application that ought to give you right performance.

Share this post


Link to post
Share on other sites
every one can say what ever they want. It still doesnt explain why my x15 36LP takes twice as long to write a 1 gig file to itself then my 10K III does in XP 

XP "bug" or not.

One thing i do believe. is a lot of peoples problumes with pour performance in XP with SCSI doesnt have to do with XP

But it sure gives for an easy scape goat. 

I sure would like to know why my x15 35LP gets trummped by my 10K III. I stopped worring about. I whent back to 2000 and I'm fixing to order a 10K IV soon.

How long does it take your X15 to copy to itself? It takes mine 20 secs to copy a 1GB file from one X15-36LP to another on the same channel, and 44 secs to copy to itself. This is using .Net Server Enterprise Server build 3718. In another post which I cannot find right now I gave the times in winXP. They were much greater (slower). There is clearly a problem with winXP Pro and SCSI performance, which has been fixed in .Net Server.

Share this post


Link to post
Share on other sites

Here's the results from XP Pro:

"OK here's the results:

TYAN TigerMP

2* AthlonXP 1700

512MB reg ecc

2*X15-36LPs (18GB) in SCA enclosure, both drives on one channel

2*WD1200AB (120GB), both on secondary IDE channel

Tekram U3D controller

WinXP Pro SP1

Copy 1GB .vob, using Windows Explorer

X15-36LP -> X15-36LP (boot drive) 25 secs

X15-36LP ->WD1200AB 35 secs

WD1200AB->WD1200AB 50 secs

WD->X15-36LP 30 secs

WD to itself 1m 42 secs

X15-36LP to itself 2m 19 secs."

As you can see, copying a file to itself, the time has been reduced massively from 2m 19 secs to 44 secs. Windows Explorer seems even snappier than win2k used to be.

Share this post


Link to post
Share on other sites

I was sifting through my disk properties for the scsi disks i currently have. It currently has write cache enabled. and it does give me that warning about data loss or corruption by power outage. I was curious. Should i uncheck it? Although what are the chances of power outtages ?

Share this post


Link to post
Share on other sites

Pradeep - I agree with you regarding .Net server. SCSI drive slowness is gone with advanced performance enabled. ATTO testing backs this up too.

Share this post


Link to post
Share on other sites

On Page 1 of this thread I wrote:

I've read through this thread and don't recollect seeing anything about this issue: since add-in IDE cards like the Promise Ultra-100 are "considered" to be SCSI controllers in Win2k/XP, are they also affected by the problems identified by Cas and Eugene?

Do we have any definite conclusions on this?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now