Eugene

Tcq, Raid, Scsi, And Sata

Recommended Posts

Intel 925 + Maxline III w/ Hardware NCQ beat Raptors!

Great drives those Raptors, but Maxtor's new 16MB HARDWARE NCQ and Intel's ICH6 w/ Hardware NCQ have at least 15% better performance on some tests.

NCQ is not the same as TCQ. FPDMA makes a huge difference!

Raptors have a 3 month window until the MaxLines become available. After that, they will suck big time (for the price).

Why TCQ is slower than non-TCQ? ... easy: TCQ is slow software!. TCQ reminds me of 3DNow! and Glide... Things that were pure marketing and then were abandoned.

Waiting for farware TCQ implementations? forget it!. I guess TCQ developement is nonexistent by now. Everybody is developing NCQ solutions. TCQ is not a long-term solution.

Please test!

My wish: NCQ drives from WDC

(Am I wrong?)

:)

Share this post


Link to post
Share on other sites
TCQ (the 'parallel ATA tagged command queueing') does _not_ need a special controller! It requires a TCQ capable hard disc and a TCQ capable (SW) driver.

Considering that the driver comes with the controller, I think this is a rather silly point, no offence. Why bother splitting this hair? In all practicality you have to buy a controller that supports TCQ.

FreeBSD has (had) a driver that supported ATA TCQ on all correctly-functioning drives. “All†meant “one†(1). Moreover, ATA TCQ has been removed from FreeBSD so I suspect “all†turned out to be “noneâ€. But no card driver is needed unless to base OS driver is missing TCQ support, and the card driver’s TCQ support would work equally well with any ATA controller is not otherwise blocked.

(in all reality it might be better to say you have to buy a drive that supports TCQ, and in the ATA world there are none)

PS. I've read the TCQ specs in the ATA document and it's pretty straightforward as long as there is only one device on the channel. I suppose it’s time to find the sATA spec and see how NCQ works. Has anyone here really dug into NCQ yet?

Share this post


Link to post
Share on other sites

Eugene: is there evidence, or has WDC said, as to whether the drive returns an early SUCCESS for NCQ writes?

I think it is important that the drive not return SUCCESS until an NCQ write has, in fact, successfully committed to media.

Existing ATA drives return a false SUCCESS and re-order writes to media without the OS’s knowledge. This can cause disk corruption that filesytem hardening strategies normally prevent. MS has published KB articles on the problem. I think the log-based nature of NTFS can help reduce the problem but more-modern filesystems that use no log are at even greater risk of damage from false SUCCESS result.

The usual reason for this sad state of affairs is performance: internal write reordering wins significantly, enough to outweigh data loss (for drive manufacturers anyway). But with NCQ this is no longer necessary: the drive may continue to accept writes and reorder them at will, suffering no performance hit from only completing the transaction (returning a result code) after data has been committed to media. The OS simply withholds ordered writes until previous activity completes when ordering is important.

This data loss issue is what really sets SCSI apart from IDE in my mind. I would promptly switch to sATA / NCQ in the drive vendors promised not to report false SUCCESS for NCQ writes with the intended benefit being crash resistance. The data loss problem probably deserved more attention in your paper; if nothing else it is a significant reason NCQ would be desirable even for a single-user workstation.

PS. Non-NCQ writes will probably continue to return false SUCCESS as before for performance reasons. It’s the NCQ write commands, which won’t benefit from cheating, that I’m hoping to see work right.

Share this post


Link to post
Share on other sites

Is it me, or is the image on the first page wrong...

Shouln't B be C and C be B for the statement to be true? In a normal rotation, the disk will see A, then B, then C, without the need of reordering, if one looks at the current image...

Share this post


Link to post
Share on other sites

<checks>

No, the image is right. Note that the arrow shows the disk platter rotating anti-clockwise - you could also visualise this as the heads moving clockwise round the platter.

That said, is there an industry standard for the direction of rotation of the platters in a hard disk, when viewed top-down? A simple yes or no will do, I realise the question is a little off-topic.

Share this post


Link to post
Share on other sites
Existing ATA drives return a false SUCCESS and re-order writes to media without the OS’s knowledge.

Don't SCSI drives do the same if the Force Unit Access bit is off?

I think you can force this behavior but there is no reason to do so if SCSI TCQ is used, and nobody in their right mind would do such a thing in a production environment.

Share this post


Link to post
Share on other sites
That said, is there an industry standard for the direction of rotation of the platters in a hard disk, when viewed top-down? A simple yes or no will do, I realise the question is a little off-topic.

Not strictly speaking, but Maxtor is the only HDD maker whose drives rotate CW when viewed from the top. The actuator and VCM are repositioned accordingly.

Share this post


Link to post
Share on other sites
That said, is there an industry standard for the direction of rotation of the platters in a hard disk, when viewed top-down? A simple yes or no will do, I realise the question is a little off-topic.

Not strictly speaking, but Maxtor is the only HDD maker whose drives rotate CW when viewed from the top. The actuator and VCM are repositioned accordingly.

Is there an advantage to doing this, or were they just being rebellious? :D

Share this post


Link to post
Share on other sites
You can disable TQ in your device manager, under disk drives.  If you have XP, find the drive and double-click on it.  The option is on the "SCSI Properties" tab.

unfortunately, eugene found this did not actually work in testing. it stays enabled on the ata tcq-enabled adapters.

I'm getting confused, he's talking about TQ, you're talking about TCQ. Now, I thought the former was for SCSI the later for ATA... or are they actually one and the same? If not, is it impossible to disable both TQ and TCQ within Windows or only one or the other? Is this because the box is broken, or because the firmware overrides the OS?

Share this post


Link to post
Share on other sites

Eugene, you mentioned in the article that any non-braindead controller will optimize reads on a RAID1 volume by spreading them across the spindles. Do you have any idea what controllers fall into the category of intelligent? I'm currently using onboard Silicon Image SATA for a pair of raptors, and I'm considering mirroring but I don't want to do so if the controller is too dumb to spread the read load.

If I have to go buy a Promise controller I may, but it would be cool to know a list of cards/controllers that do RAID 1 "the right way."

Share this post


Link to post
Share on other sites

I'd still like to hear from someone in the know why NCQ seems to have single-user benefits while TCQ does not. Or if there was some methodological problem with TR's and Anand's reviews (easy to believe with Anand... :rolleyes: ).

Share this post


Link to post
Share on other sites
I'd still like to hear from someone in the know why NCQ seems to have single-user benefits while TCQ does not.  Or if there was some methodological problem with TR's and Anand's reviews (easy to believe with Anand... :rolleyes: ).

Where are TechReport's single-user benchmarks? They used only IOMeter, which represents multi-user instances only.

Share this post


Link to post
Share on other sites
If I have to go buy a Promise controller I may, but it would be cool to know a list of cards/controllers that do RAID 1 "the right way."

I doubt if Eugene has such a list...perhaps one of the HDD manufacturers could give you a list of some sort?

Share this post


Link to post
Share on other sites
You can disable TQ in your device manager, under disk drives.  If you have XP, find the drive and double-click on it.  The option is on the "SCSI Properties" tab.

unfortunately, eugene found this did not actually work in testing. it stays enabled on the ata tcq-enabled adapters.

I'm getting confused, he's talking about TQ, you're talking about TCQ. Now, I thought the former was for SCSI the later for ATA... or are they actually one and the same? If not, is it impossible to disable both TQ and TCQ within Windows or only one or the other? Is this because the box is broken, or because the firmware overrides the OS?

Can anyone answer me this? Are TQ and TCQ one and the same? Do both suffer from the same problem within Windows regarding this checkbox?

Share this post


Link to post
Share on other sites
I'd still like to hear from someone in the know why NCQ seems to have single-user benefits while TCQ does not.  Or if there was some methodological problem with TR's and Anand's reviews (easy to believe with Anand... :rolleyes: ).

Don't get too fixated on raw I/O performance. If the drive and OS driver both work right then both TCQ and NCQ should make the system much less likely to corrupt the hard disk than ordinary ATA.

Getting TCQ/NCQ to the desktop is about data integrity (hardening the filesystem against data loss) and not performance.

Share this post


Link to post
Share on other sites
Don't get too fixated on raw I/O performance.  If the drive and OS driver both work right then both TCQ and NCQ should make the system much less likely to corrupt the hard disk than ordinary ATA.

Getting TCQ/NCQ to the desktop is about data integrity (hardening the filesystem against data loss) and not performance.

How does it do that?

Can anyone answer me this? Are TQ and TCQ one and the same? Do both suffer from the same problem within Windows regarding this checkbox?

I'm not really sure that there is any such thing as "TQ." There's "TCQ" (ATA and SCSI versions) and "NCQ" (SATA version), but the only reference I've found to "TQ" is in an X-bit Labs' article on the 74GB Raptor. In this article, it's pretty clear that "TQ" is just another way of saying "CQ" or "TCQ."

My guess is that you don't have to worry about "TQ." It's just TCQ.

Share this post


Link to post
Share on other sites

There is some more information about TCQ overhead in this Intel paper about Native Command Queuing:

ftp://download.intel.com/design/storage/papers/25266401.pdf

Parallel ATA Queuing Overview

Parallel ATA Queuing is a queuing protocol that was added to the ATA specification in 1997.Today, only one drive vendor is shipping drives that support this queuing protocol because of the high overhead. The high overhead of Parallel ATA Queuing causes it to significantly underperform non-queued commands when the queuing workload is light (especially when the protocol is not fully hardware accelerated). The Parallel ATA Queuing protocol consists of several commands including Read DMA Queued (Ext), Write DMA Queued (Ext), and Service.

The significant overhead of Parallel ATA Queuing is due to a split command issue and data transfer phase. Before the data transfer for a Parallel ATA queued command, the host must issue the Service command to determine which command the data transfer is for, and then the host will set up the DMA engine for the appropriate command. In order to alert the host to issue the Service command, an additional interrupt is taken per IO. There is indeterminate latency for the host software to take control and issue the Service command if the protocol is not fully hardware accelerated. This latency is the major cause for the poor performance of Parallel ATA Queuing.

[...] Command Queuing Comparison

To summarize, Parallel ATA Command Queuing has significant overhead (especially when there is no hardware acceleration) that impacts its performance appreciably. Serial ATA II Native Command Queuing eliminates the shortcomings of Parallel ATA Command Queuing and achieves a queuing protocol similar in overhead to SCSI queuing.

Share this post


Link to post
Share on other sites

Eugene,

there is what looks to be a copy and paste error in the article. In the table comparing the 2nd and 3rd Samples of the Raptor, the Web Server Drive Marks are incorrect. They should be 210 and 208 respectively. The current erroneous values (188 and 189) are the respective entries for the FS DriveMarks at 64 queue depth.

Share this post


Link to post
Share on other sites
Eugene, the current erroneous values (188 and 189) are the respective entries for the FS DriveMarks at 64 queue depth.

Fixed, thanks for the sharp eye and correction!

Share this post


Link to post
Share on other sites

SR team,

I first want to say thank you for your in-depth single user vs. multi-user RAID analysis. It is very clear why RAID0 has little benefit for the single user. I was wondering if you have done any analysis on the actual performance gains of multiple non-raid disk configurations in desktops? For instance, placing the swap or pagefile on a separate physical drive from the system and/or putting the data being frequently accessed on yet another drive. Multi-user servers (especially database servers) gain a lot from reading/writing simultaneously to separate spindles / disk arrays. So my question is: does the same hold true for the single user scenario under heavy load? Under low memory conditions? Working with lots of monstrously large files that can’t fit into RAM? This would seem to be a better approach to leveraging multiple hard drives in the same desktop. Would you agree?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now