Ok, so this time around, I wrote a small test app, and used it to test things.
FlushFileBuffers() API definitely seems to work correctly, even with the write-caches all enabled. I see 1 write per 3 disk rpm on the laptop, and per 2 disk rpm on the desktop.
(ie, 30 writes/sec on the laptop, and 250 writes/sec on the desktop)
however, in the cases of Unbuffered or WriteThroughMode filemodes, it's an entirely different story. I can read the drive rpm directly on the SCSI box. One write per rpm, perfectly. I get insanely high writes/sec on the laptop.
If I disable disk write-caching on both, I see that that performance ever so slightly drops on the SCSI (to 14800 writes/min), and the laptop starts showing 5200 writes/min, which makes sense for a 5400 rpm drive.
Now... I'd say in the cases of the unbuffered and write-through modes, the laptop isn't behaving correctly. While it's obvious doing so in the FlushFileBuffers situations.
An amusing note is that the best flushed performance comes from using unbuffered files with flushes. Perhaps because it's already starting to write to disk when the flush comes through? or maybe already pushed lower the stack of caches... I don't know.
The writes were done by opening a single file, and then in a tight loop seeking to the start of the file, writing 1K to it, Flushing, seeking to start, etc.
Relevant win32 api calls are:
SetFilePointer(file_handle, 0, 0, FILE_BEGIN);
WriteFile(file_handle, buffer, 1024, &amount_written, NULL);
Anyone wants a copy (binary or source), let me know.