6_6_6

NCQ: Best Upgrade For a Power User!

Recommended Posts

Its strange how the Maxtor drives had such good non-NCQ performance. Even the modern Seagate drives perform worse when not running in AHCI mode. Were you thinking that they implemented an NCQ-related performance jump even when the drive is run under IDE mode?

Share this post


Link to post
Share on other sites
I wonder then why neither of the companies mentioned increased NCQ performance?

Because there is NO NCQ performance increase. We have been using NCQ and AHCI interchangeably and this is wrong. Seagate does not do any better NCQ than other drives according to my linux and IOmeter tests.

However, its firmware behaviour in AHCI mode does wonders as far as performance and multitasking is concerned. All our tests here were related to AHCI... not a single one for NCQ. Unfortunately, I cannot change thread title but i mentioned the revised title few posts back.

Share this post


Link to post
Share on other sites
Because there is NO NCQ performance increase. We have been using NCQ and AHCI interchangeably and this is wrong. Seagate does not do any better NCQ than other drives according to my linux and IOmeter tests.

Ah, but NCQ is only available in AHCI mode, and while in AHCI mode there are specific NCQ type performance increases in Windows. Look at the full read table results from X-bit labs IOmeter tests, and the 7200.11 stomps all of the other drives for a queue depth of 2-8 deep, for 1-4 threads. It doesn't do too shabby on the write tests either.

Certainly the specific causes are a mystery, but it isn't a bad guess to say that Seagate may have tuned their NCQ performance specifically for Windows loads.

Why doesn't someone use a Seagate 7200.11 drive in AHCI mode and test the results with NCQ turned on and off? (You can turn NCQ for a drive on and off, no?)

Share this post


Link to post
Share on other sites

I could make some tests at work. We have recently bought a pair of 500GB Seagate 7200.11 drives and have enclosed them in removable bays to transport them easily.

The server is an HP Proliant ML150 G3, and it's using a PCI-X LSI Logic controller (SAS3080X-R) with 8 internal SAS/SATA ports. There's a small command line tool called "lsiutil" that can be used to enable/disable NCQ selectively in each drive.

What exactly do you want to test?

Share this post


Link to post
Share on other sites
Because there is NO NCQ performance increase. We have been using NCQ and AHCI interchangeably and this is wrong. Seagate does not do any better NCQ than other drives according to my linux and IOmeter tests.

Ah, but NCQ is only available in AHCI mode, and while in AHCI mode there are specific NCQ type performance increases in Windows. Look at the full read table results from X-bit labs IOmeter tests, and the 7200.11 stomps all of the other drives for a queue depth of 2-8 deep, for 1-4 threads. It doesn't do too shabby on the write tests either.

Certainly the specific causes are a mystery, but it isn't a bad guess to say that Seagate may have tuned their NCQ performance specifically for Windows loads.

Why doesn't someone use a Seagate 7200.11 drive in AHCI mode and test the results with NCQ turned on and off? (You can turn NCQ for a drive on and off, no?)

I could make some tests at work. We have recently bought a pair of 500GB Seagate 7200.11 drives and have enclosed them in removable bays to transport them easily.

The server is an HP Proliant ML150 G3, and it's using a PCI-X LSI Logic controller (SAS3080X-R) with 8 internal SAS/SATA ports. There's a small command line tool called "lsiutil" that can be used to enable/disable NCQ selectively in each drive.

What exactly do you want to test?

Leolo: I think Atamido is asking someone to establish, : 'is it actually 'just NCQ' that is responsible for the multitaksing advantages of these drives or is it 'just/also AHCI'.

We are having difficulty finding out if 'NCQ technology' having an effect, or is it 'AHCI' or maybe even the combination.

Could you possibly do tests on this drives using AHCI mode only - with NCQ turned off. And then do tests with NCQ without AHCI (have I got this right anyone?), and then one without both, then one with both.

This way you could finally solve the dilemma of almost 11 full pages and months of competition.

Could anyone (666 / Anyone) detail exactly the tests/the programs and the method someone would use to do this - it would make life easier (I suspect) and we can finally have answers.

LittleJohn

Share this post


Link to post
Share on other sites
If you're running XP, then you should set SATA MODE to "IDE". Doing that disables AHCI, which in turns also disables NCQ (remember that NCQ can only be enabled in AHCI mode or RAID mode).

If you want to get the highest performance possible under Windows XP you need the following tool:

(But be careful not to install the 332023 update as instructed by their web page. That update is really designed only for Windows 2000!!)

http://support.microsoft.com/kb/811392

Click on "View and request hotfix downloads" to get your copy. It's named Windows2000-KB811392-x86-ENU, but don't be fooled by the file name. It works correctly also in Windows XP, 2003, Vista and Server 2008.

PS: Gotta love Microsoft's inconsistencies and all of their confusing patches and file names, heh ;)

Leolo - this is really interesting. If you could explain what this does, and why it is good - would be great. The website goes into so much detail - that the result is hard to understand.

Please talk a little about what this is about and what the nature of the results would be. Thanks,

LittleJohn.

Share this post


Link to post
Share on other sites

Well, Microsoft "explanations" make it look more difficult than it really is.

In Microsoft's parlance, "Write Cache enabled" means Write-Through cache enabled

And "Power Protect enabled" means Write-back cache enabled

Write-back cache is the fastest, but it's also the most dangerous. In this mode Windows won't write the data to your hard disk immediately, it will instead wait until the drive is idle and isn't busy doing other things. Performance will be higher, but you run the risk of losing data if you suffer a power loss.

To overcome this problem you have three options:

1) Buy a UPS system to protect you from power cuts

or

2) Buy an extremely expensive controller that includes an internal battery.

or

3) Buy both an UPS and an extremely expensive controller if you want to be super-extra safe.

I'm not rich, so I've opted for option 1 ;)

Regards.

Share this post


Link to post
Share on other sites

Write-cache setting cannot be modified on any of the new drives (i tested plenty) since all come with write-cache enabled by default. That cannot even be changed with ATA commands.

That option is useless. Checking/unchecking that box won't do a thing. It is a thing of the past.

Well, Microsoft "explanations" make it look more difficult than it really is.

In Microsoft's parlance, "Write Cache enabled" means Write-Through cache enabled

And "Power Protect enabled" means Write-back cache enabled

Write-back cache is the fastest, but it's also the most dangerous. In this mode Windows won't write the data to your hard disk immediately, it will instead wait until the drive is idle and isn't busy doing other things. Performance will be higher, but you run the risk of losing data if you suffer a power loss.

Share this post


Link to post
Share on other sites
Write-cache setting cannot be modified on any of the new drives (i tested plenty) since all come with write-cache enabled by default. That cannot even be changed with ATA commands.

That option is useless. Checking/unchecking that box won't do a thing. It is a thing of the past.

Official specs don't agree with you. They say it CAN be changed. Take a look at this official document about ATA commands:

http://www.t13.org/Documents/UploadedDocum...4a-ATA8-ACS.pdf

7.48.4 Enable/disable write cache

Subcommand codes 02h and 82h allow the host to enable or disable volatile write cache in devices that

implement volatile write cache. When the subcommand disable volatile write cache is issued, the device shall

initiate the sequence to flush volatile cache to non-volatile media before command completion (see 7.14). These

subcommands may affect caching for commands in the Streaming feature set.

Also, look at these two official links from Microsoft. They explain how to query for the presence of a write cache:

http://msdn.microsoft.com/en-us/library/aa508863.aspx

http://msdn.microsoft.com/en-us/library/aa510119.aspx

How did you conduct your testing? Which drives refused to disable the write cache? From what manufacturer? I think you must have done something wrong with your tests.

Regards.

Edited by Leolo

Share this post


Link to post
Share on other sites

I've searched a bit more about the topic and found this link:

http://weblogs.java.net/blog/davidvc/archi...tory_of_th.html

Do you know if this is a widespread practice or is it just a few isolated cases?

I don't understand why would a drive manufacturer violate the ATA specs so blatantly.

Well, perhaps I do. It's another way to justify the higher cost of SAS drives. Because disabling write caching is important for servers, they will be able to mark this feature as another selling point.

So, now if you want the command to disable the cache to work reliably, you will have no choice but to buy a SAS drive!

Sigh... :(

Share this post


Link to post
Share on other sites

> I don't understand why would a drive manufacturer violate the

> ATA specs so blatantly.

No makes or models were given, so there is no way to prove/disprove

this rumor.

Controllers are another story. I have yet to find a USB-to-[PS]ATA

bridge that will allow you to turn the write cache on and off.

Share this post


Link to post
Share on other sites

Hi Konrad,

Let's hope it's just a rumor. It would be quite sad if manufacturers were doing that crime with premeditation.

I've found another interesting link:

http://www.bitmicro.com/app_notes/APN-MK-0...che_Disable.pdf

Granted, the link above it's only for flash SSD drives, but... what if traditional mechanical hard disk manufacturers were doing the same dishonest trick? Man, I think there's grounds here for a lawsuit if it's true!!

Regards.

Share this post


Link to post
Share on other sites

What lawsuit? Aren't you ever satisfied? Drive manufacturers implemented faster performance in their drives and saved you from trying to modify OS/controller, etc and you are still bashing them.

Most modern filesystems are robust. Most widely used fs, NTFS with v6.0 in Vista/2008 implemented transactional NTFS. File system corruption is a thing of the past. I have pulled the plug on benchmarking systems plenty of times and i am yet to see corruption on my filesystem last 2-3 years.

Share this post


Link to post
Share on other sites

Modern filesystems are robust?!? What!!? What does "robust" mean for you?

I think you may be referring to Sun ZFS, which does have checksums for checking data integrity. In that case, yes, I agree with you that a modern filesystem like ZFS is robust.

But what about NTFS? NTFS does not have any kind of data integrity checking. It doesn't even have simple CRC checking!

NTFS permits silent data corruption and doesn't tell you anything about it. That's about the worst imaginable kind of corruption, because you will not notice it and you will happily keep going thinking that everything is alright. By the time you discover the error, it's probably too late and you'll have overwritten your backups with more recent (and silently corrupted) data.

Please, read these two links:

http://blogs.zdnet.com/storage/?p=169

http://blogs.zdnet.com/storage/?p=191

Man, I'm asking you seriously. Please answer me with a straight face. Do you really think that NTFS is robust?

Edited by Leolo

Share this post


Link to post
Share on other sites

Hi again,

Regarding the rumor about drives ignoring the "disable cache" command, I've done a few tests today.

Hardware specs:

- HP Proliant ML150 G3

- LSI HP3080X-R SAS controller

- Seagate Barracuda 7200.11 SATA (500 GB)

Software specs:

- Windows 2003 Standard 32 bit with SP2

- LSI drivers v1.28.3

- Microsoft Robocopy v. XP010

First I copied a directory with a few big files to the Seagate drive with the cache disabled (via Device Manager in Windows):

-------------------------------------------------------------------------------
  ROBOCOPY	 ::	 Robust File Copy for Windows	 ::	 Version XP010
-------------------------------------------------------------------------------

 Started : Wed Nov 19 14:00:42 2008

  Source : i:\imagenesdocumentum\
 Dest : j:\imagenesdocumentum\

Files : *.*

 Options : *.* /COPY:DAT /R:1000000 /W:30

------------------------------------------------------------------------------

					   4	i:\imagenesdocumentum\
100%		New File			 649.9 m		IMAGENE.304
100%		New File			 649.9 m		IMAGENE.305
100%		New File			 649.9 m		IMAGENE.306
100%		New File			  57.5 m		IMAGENE.307

------------------------------------------------------------------------------

			Total	Copied   Skipped  Mismatch	FAILED	Extras
 Dirs :		 1		 0		 1		 0		 0		 0
Files :		 4		 4		 0		 0		 0		 0
Bytes :   1.960 g   1.960 g		 0		 0		 0		 0
Times :   0:00:37   0:00:37					   0:00:00   0:00:00

Speed :			56697377 Bytes/sec.
Speed :			3244.249 MegaBytes/min.

Ended : Wed Nov 19 14:01:19 2008

Afterwards, I enabled the writeback cache (again, using the Device Manager in Windows):

-------------------------------------------------------------------------------
  ROBOCOPY	 ::	 Robust File Copy for Windows	 ::	 Version XP010
-------------------------------------------------------------------------------

 Started : Wed Nov 19 13:58:01 2008

  Source : i:\imagenesdocumentum\
 Dest : j:\imagenesdocumentum\

Files : *.*

 Options : *.* /COPY:DAT /R:1000000 /W:30

------------------------------------------------------------------------------

					   4	i:\imagenesdocumentum\
100%		New File			 649.9 m		IMAGENE.304
100%		New File			 649.9 m		IMAGENE.305
100%		New File			 649.9 m		IMAGENE.306
100%		New File			  57.5 m		IMAGENE.307

------------------------------------------------------------------------------

			Total	Copied   Skipped  Mismatch	FAILED	Extras
 Dirs :		 1		 0		 1		 0		 0		 0
Files :		 4		 4		 0		 0		 0		 0
Bytes :   1.960 g   1.960 g		 0		 0		 0		 0
Times :   0:00:29   0:00:29					   0:00:00   0:00:00

Speed :			70754042 Bytes/sec.
Speed :			4048.578 MegaBytes/min.

Ended : Wed Nov 19 13:58:31 2008

Conclusion: The Seagate 7200.11 drive does honor requests to disable the cache.

666, I'm asking you again, could you tell us which drives ignore requests to disable the cache??

It's important to know which ones do that. I want to know what drives to avoid.

Regards.

Share this post


Link to post
Share on other sites

Ooops! I misstyped the previous post. Just a correction: I did the test with the cache enabled first, and afterwards I did the test with the cache disabled.

The results aren't affected, though. The conclusion is the same: The Seagate Barracuda 7200.11 drive does disable the cache when it's asked to.

Regards.

Edited by Leolo

Share this post


Link to post
Share on other sites
But what about NTFS? NTFS does not have any kind of data integrity checking. It doesn't even have simple CRC checking!

CRC checking by filesystem? Must surely be good for the same kind of embeciles running 5 different real-time antivirus programs checking each file access/read/copy/write... or who need to answer 10 UAC prompts to do a simple file copy... or for military grade institutions... but no, thanks, i will pass. I surely would detest any filesystem of this type.

NTFS permits silent data corruption and doesn't tell you anything about it. That's about the worst imaginable kind of corruption, because you will not notice it and you will happily keep going thinking that everything is alright. By the time you discover the error, it's probably too late and you'll have overwritten your backups with more recent (and silently corrupted) data.

As one user pointed out, this silent data corruption can be summarized with "Blame software for the blown bearings on the platter motor"! Manufacturers state error rates adn that is expected. NTFS or no NTFS, you are getting the same corruption with other filesystems as well. My important data has md5sums. I surely do not expect NTFS saving me from a failing drive head! If you are expecting that, good luck!

Share this post


Link to post
Share on other sites
666, I'm asking you again, could you tell us which drives ignore requests to disable the cache??

It's important to know which ones do that. I want to know what drives to avoid.

That same Seagate and many others i tried. All have write-cache enabled by default and that cannot be changed neither through OS nor through ATA commands.

And i like it that way. And i see no point in discussing how to cripple my drives.

Share this post


Link to post
Share on other sites

666,

I'm afraid that you did your tests wrong. You should redo them properly in order to get correct results.

Take a look at these results. I've made them at home.

Hardware specs:

ASUS P5B-E mainboard (bios 1803)

intel Core2Duo E6600

Seagate ST3500320AS drive (firmware SD15)

Software specs:

Windows Vista Ultimate 64 bit SP1

Microsoft native AHCI drivers

HD Tune Pro 3.10

Results with write cache ENABLED:

enabledcachesettings.png

enabledcache.png

Results with write cache DISABLED:

disabledcachesettings.png

disabledcache.png

It's quite obvious that the drive is obeying the order to disable the cache. Look at the performance drop!

What kind of results did you get? How did you do the benchmarks?

Regards.

Share this post


Link to post
Share on other sites
CRC checking by filesystem? Must surely be good for the same kind of embeciles running 5 different real-time antivirus programs checking each file access/read/copy/write... or who need to answer 10 UAC prompts to do a simple file copy... or for military grade institutions... but no, thanks, i will pass. I surely would detest any filesystem of this type.

You would detest a filesystem that could detect silent data corruption? Well, to each his own. I would love such a filesystem. In fact, I admire Sun's ZFS because of that.

Anyway, you can always disable the checksumming if you're so annoyed by it. Solarisinternals says that "Fletcher2 checksum (the default of ZFS) has been observed to consume roughly 1Ghz of a CPU when checksumming 500 MByte per second."

I think it's perfectly reasonable to assume such a CPU hit, it's justified by the benefits. It would alleviate the need of performing the checksumming by external applications.

As one user pointed out, this silent data corruption can be summarized with "Blame software for the blown bearings on the platter motor"! Manufacturers state error rates adn that is expected. NTFS or no NTFS, you are getting the same corruption with other filesystems as well. My important data has md5sums. I surely do not expect NTFS saving me from a failing drive head! If you are expecting that, good luck!

So, you are using an utility to generate and check md5sums for your important files? Then you're admitting that you DO care about your data and you're willing to consume CPU cycles to that task.

Then why do you oppose checksumming done by the filesystem? Why are you contradicting yourself?

Regards.

Share this post


Link to post
Share on other sites

I don't have to do any tests since my drives have write-cache enabled by default. I have no interest if write-cache can be disabled or not through software or whatever. I have no time to waste on figuring out how to cripple my drive. They are already coming exactly as i want them! I was doing some low level hd repairs, and just out of curiosity, i played with write-cache settings. Older drives permitted that to be changed, but newer drives did not.

666,

I'm afraid that you did your tests wrong. You should redo them properly in order to get correct results.

Share this post


Link to post
Share on other sites
So, you are using an utility to generate and check md5sums for your important files? Then you're admitting that you DO care about your data and you're willing to consume CPU cycles to that task.

Then why do you oppose checksumming done by the filesystem? Why are you contradicting yourself?

Same reason i do not run real-time virus scanner and run it manually when need be. Do not compare md5suming 1 gb of data a month to checksumming 100 gb of data a day. It is not my cup of tea, if you like it then do it with the fs of your choice and expect it to fix the bearings on your motor too.

Share this post


Link to post
Share on other sites
Same reason i do not run real-time virus scanner and run it manually when need be. Do not compare md5suming 1 gb of data a month to checksumming 100 gb of data a day. It is not my cup of tea, if you like it then do it with the fs of your choice and expect it to fix the bearings on your motor too.

OK. I understand your position. We just think differently, and I still don't agree with you. With a filesystem like ZFS I would very much prefer to have a dedicated partition with checksumming enabled, where I would store my important files. I would disable checksumming on the other partitions if I really wanted the extra speed. In that way I could get the best of both worlds.

The problem with your method is that you only verify checksums once a month, and there's still the possibility that you could suffer silent data corruption and don't notice it until it's too late. You could have overwritten your backups with silently corrupted data by the time you discover the error.

With ZFS and its built-in checksumming you would notice data corruption just in the very moment it occurred, so you could take action immediately.

I don't have to do any tests since my drives have write-cache enabled by default. I have no interest if write-cache can be disabled or not through software or whatever. I have no time to waste on figuring out how to cripple my drive. They are already coming exactly as i want them! I was doing some low level hd repairs, and just out of curiosity, i played with write-cache settings. Older drives permitted that to be changed, but newer drives did not.

I'm sorry, but if you don't have time to waste on figuring out how to enable or disable the write cache, then it's obvious that you won't be able to do the tests correctly.

In that case, I would politely ask you to stop spreading misinformation and confusing people.

Regards.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now