• Content Count

  • Joined

  • Last visited

Community Reputation

0 Neutral

About woody77

  • Rank
  1. woody77

    SCSI vs SATA, Which is Faster?

    My response to the question would be: fastest at what? Sustained Transfers? Seek times? And is data integrity important? Sustained transfer rates are easy to find, and are well documented by the drive manufacturers. Same with seek times. But seek times are a function of spindle speed. A drive's worst-case seek time should be no more than 1-2rev of the spindle. That's normally about the way it works out. Higher density drives have better average rates because on average, they aren't moving as far, but that really depends on where you're seeking too and where you're at now. Sequential operations don't involve much in the way of seeking, unless you're copying a file to/from the same partition, then you need to seek between the two locations, and the faster seek times come into play again. Data Integrity: IDE drives (PATA/SATA) tend to ship with an enabled write-back cache, and some don't allow it to be disabled. This is cheating, unless the drive is capable of saving it's cache on a power failure (I've heard rumors of using capacitance and the spindle wind-down to keep the drive powered long enough to flush cache...) SCSI drives don't do this. But if you pay for a controller with a battery-backed write-back cache, then you can enable it, and you're good to go. using a write-back cache way speeds up small writes. So, again, faster at what?
  2. woody77

    Tcq, Raid, Scsi, And Sata

    At my previous job I did a lot of comparisons of caching, raid, and disk-drive type performances for estimating which configurations would provide the best application response for our product, which incorporated multiple databases. One database used a single large file, and that was very much unfazed by the differing hardware I tossed underneath it. The other database used many, many, many separate files. Transactions would flush to disk at start and finish. Guarantees of solidly written data, but at a massive expense in performance. In the end, what I found mattered was spindle rpm. 15K drives have faster seek times, end of story. I had a 5400 rpm PATA-133 drive that was spanking the 15K rpm scsi-160 drive. It was lying about writing data to disk, and was caching it instead. After using the utility from MS to turn of write caches, I got the expected spindle-speed limited performance on writes. However, with flushing multiple files to disk, the command queuing can make a big difference, as it can order the file flushes correctly, as multiple small files being accessed simultaneously start to look more like a "multi-user" scenario. I'd be interested in seeing similar tests done with real raid systems, though. RAID5, on a real controller, and with controllers that can do write-through caching and write-back caching, and use battery-backed ram for cache, so that they can respond to the filesystem, and work the disks as needed later. But then, this was more an article about the TCQ and NCQ, etc, so it's going to be about I/O, and not about throughput of data.
  3. Are you worried about the IDE bus power consumption, or just the power consumption through the 4-pin connector? The 4-pin should be easy, but again, probably involves running a multimeter through, and that has issues where head seeks are going to be pulling spikes of current through the lines that won't be well tracked on the MM. One solution is to build a small integrating circuit that determines the total power consumed by the device over a period of time (again, just over the 4-pin power connector for the 3.5" drive). If you wanted to get fancy, you could probably build a multiple-channel power meter on the data-bus (one for each non-ground I/O line), and then sum all those with the 4-pin power, and then integrate that to get the total consumption during the tests. Easy to provide a "reset" button that resets the measurement back to 0 for restarting tests. I wish there was an easy way to get data into a PC. Then this could all be logged and possibly even controlled by the test-software that's doing the benchmarking.
  4. Ok, so this time around, I wrote a small test app, and used it to test things. FlushFileBuffers() API definitely seems to work correctly, even with the write-caches all enabled. I see 1 write per 3 disk rpm on the laptop, and per 2 disk rpm on the desktop. (ie, 30 writes/sec on the laptop, and 250 writes/sec on the desktop) however, in the cases of Unbuffered or WriteThroughMode filemodes, it's an entirely different story. I can read the drive rpm directly on the SCSI box. One write per rpm, perfectly. I get insanely high writes/sec on the laptop. If I disable disk write-caching on both, I see that that performance ever so slightly drops on the SCSI (to 14800 writes/min), and the laptop starts showing 5200 writes/min, which makes sense for a 5400 rpm drive. Now... I'd say in the cases of the unbuffered and write-through modes, the laptop isn't behaving correctly. While it's obvious doing so in the FlushFileBuffers situations. An amusing note is that the best flushed performance comes from using unbuffered files with flushes. Perhaps because it's already starting to write to disk when the flush comes through? or maybe already pushed lower the stack of caches... I don't know. The writes were done by opening a single file, and then in a tight loop seeking to the start of the file, writing 1K to it, Flushing, seeking to start, etc. Relevant win32 api calls are: SetFilePointer(file_handle, 0, 0, FILE_BEGIN); WriteFile(file_handle, buffer, 1024, &amount_written, NULL); FlushFileBuffers(file_handle); Anyone wants a copy (binary or source), let me know.
  5. And the smoking gun, is yes, the write-back cache on XP. For an IDE drive, go to the Properties for it in the Hardware Manager, and then to the Policies tab. Uncheck the "Enable Write Caching on the Disk" check box. The one with the description: "This setting enables write caching to improve disk performance, but a power outage or equipment failure might result in data loss or corruption." This is checked by default, btw. Our application's performance went from 50ms/operation to 350ms/operation. Toggle it while we're running, and it's amazing the difference. joy, joy joy... Thanks for this forum providing the clues to stumble across this.
  6. found a link in the forum here for this, appears I'm recovering old territory. Sorry about that. Missed those threads from back then.... http://forums.storagereview.net/index.php?showtopic=8865 I've ran the dskcache utility, and enabled the PowerProtected flag on the scsi drives, and after re-running my test-applicaiton, I'm seeing some better performance numbers. Although I'm really surprised that there's this discrepancy between SCSI and IDE on the handling of this. If the IDE drives are actually flushing their write buffers, then it seems like they should be just as slow as the SCSI drives... And after using the dskcache util, I'm seeing 135ms operations on the desktop with SCSI, which is still considerably slower than the 50ms operations on the laptop, but a lot better than the 250ms operations I was seeing before. However, I'm not happy about the change in reliability that I had to make to get it... Although there may be differences between platforms, and between chipsets that are making the rest of the problem murky. Best bet at this point is probably to order a few HDs (SCSI and ATA/100 drives, and maybe see if I can get access to a U320 RAID5 capable card and an SATA card, and play around with disk configurations, and see what's the best for our situation). Thanks for your help.
  7. relevant Knowbase articles from MS: Why It's Slow, and options for fixing it (On Win2K): http://support.microsoft.com/default.aspx?...kb;en-us;332023 Where to get the utility to specify that the disks are Power Protected: http://support.microsoft.com/default.aspx?...kb;EN-US;811392 However, you can't seem to be able to download the stupid utility.... Anyone have it?
  8. Thanks, ericg. I'm seeing it now. This is further proof, however, that we should be running SCSI instead of IDE for the system... I'm not sure exactly what controls we have for flushes vs. unbuffered I/O. I'll poke around for the whitepaper. Unfortunately, we're a developer, and don't setup the final PCs that use our system (which includes multiple databases). However, we can make recommendations for squeezing the performance out of it. I did determine that there was no performance difference on the laptop between NTFS and FAT32, but I don't have that option on the desktop (at least not without another HD). Although I think I'm about to go see if I can borrow a few U160 HDs and play with dynamic disks, although I understand that dynamic disks are currently not using the FUA flag correctly (on XP only?). This from cas' thread on the Dangerous XP Cache Filter. And I found HD Tach (not HD Bench...). SCSI devices were rocking all over the IDE in that, as well. Although I'm only getting 40MB on the sequential reads from the U160 HDD... 20MB on the U80, and 30 down to 15 on the laptop, tapering off towards the end of the disk, whereas the SCSIs stayed flat...
  9. Playing with Sandra (couldn't find a download for HDBench except for a million russian sites), and I came up with some interesting numbers: Laptop: Bypassing Windows Buffer: 67/38 MB/s Buffered Read/Write 9/13 MB/s Sequential Read/Write 2.3/4 MB/s Random Read/Write 20ms access time (est.) Enabling Windows Buffer: 64/23 4/13 5/446 <- well then.... 1ms Desktop: Bypassing Windows Buffer: 83/16 36/35 9/15 5ms Enabling Windows Buffer: 86/15 28/7 12/16 3ms Smoking gun then perhaps the WinXP filesystem buffer, and perhaps it's not exactly being honest about flushing to disk?
  10. Gah, I knew I'd screw up and leave something out. Actually, the OS's are different, but not what you'd exptect (from reading this forum and the Dell Support forum). Desktop is Win2K Pro, latest SP and patches and drivers from adaptec for the controller Laptop is WinXP Pro, latest SP and patches.
  11. I've seen the posts on SCSI throughput being slow in a lot of cases, and I've read through those. So, first I'll start with what the problem is, then what I've done, and things that I've noted: Problem: Database application getting poor transaction performance, apparently disk-limited. In comparing two different PCs, I'm getting wildly different results. PC1: Dell Precision 530 Workstation Dual PIII Xeon 1.4Ghz 512MB Memory Seagate Cheetah 15K rpm SCSI160 HDD (2MB Cache) IBM 7200rpm SCSI80 HDD (2MB Cache) Integrated to mobo Adaptec aic-7892 (aka 29160?) PC2: IBM Thinkpad T30 P4m 1.8Ghz 512MB Memory Toshiba MK4019GAX 5400 rpm ATA/100 HDD (16MB Cache) Integrated to chipset (i845MP) intel I/O controller PC1 seems to run about 250ms per transaction operation, PC2 about 50ms. CPU load is nil in either case (<5%). A transaction operation includes multiple blocking writes to several different files (journals, dbs, etc.). Blocking as in we wait for a Flush to return before moving on. Things I've done Now, I've poked at things a bit, and tried the following: - Enabling write-caching on the SCSI controller = no change - New cabling/terminators = no change - Switching the databases between the two SCSI drives = no change. Testing I've ran some benchmarks on the drives, on both PCs. Specifically using iometer. Using that, I see 8ms average I/O response time on both the laptop and the Cheetah. I see a bit slower on the IBM, at 12ms). Nothing I can do in the benchmarking software can seem to get the laptop to show this kind of a performance advantage over the desktop. Speculation My only thoughts are that it's due to the simpler design of the laptop's I/O system giving reduced latency. The I/O controller hangs right off the processor bus, and directly talks to the drives, and the CPU does all the work. On the Desktop, the aic7892 is off the PCI bus, off the processor bus, and the drive is the one that's replying back that the write is complete, instead of an I/O controller on the processor bus, so much less latency during the flushes? Latencies that the benchmark doesn't see because it's not doing them? Or is there some caching going on with the flushes on the laptop that isn't happening on the desktop, and that's causing the flush to return before the data's actually written? HDD noise lends some weight to the last. The laptop HDD isn't "working" as much as the SCSI drives are when they're being tested, with respect to the database. About the same in the benchmark tools. But the very quiet laptop HDD is much harder to hear than the authoritative chunking of the SCSI drives.... Anyone have any suggestions, or thoughts?