johnw42, on 28 March 2012 - 01:19 PM, said:
Great job on the steady-state portion of the review!
But there were two major problems, the first is by far the worst:
1) Terrible choice of comparison SSDs. All 4 competitors were nearly equivalent Sandforce SSDs! What were you thinking?
The competitors should have been: Intel 520, one other Sandforce SSD (Vertex3, HyperX, or KC100), Samsung 830, Crucial m4, and Intel 320.
2) Your "real world" tests are flawed, and not at all representative of real world usage. For example, look at the "HTPC disk capture" test. Video files are a perfect example of incompressible data. And yet your Sandforce SSDs report rates of 450MB/s, which is close to the easily-compressible sequential write rates you measured with IOMeter of 460MB/s, and nowhere near the incompressible sequential write rates of less than 300MB/s you measured with IOMeter. Clearly your "real world" test is writing UNrealistic highly compressible files instead of real video files.
We are ever so close to getting a dynamic chart generation system in progress (actually using it manually now) and with that in place it will be easier to handle comparables. This we used the most current set with the M3S and M3P added on.
On the real-world trace aspect, it is still working with direct LBA addressing, the next iteration can apply specific datatypes (fully incompressible/compressible). With it requiring a full retest of every drive to be meaningful at launch, we will be introducing it on consumer drives when we launch out next testing platform that is in validation now. Direct LBA interaction is still a step forward from generic I/O from a synthetic benchmark... and addressing the newer SSD controllers is a step we already have in place ready to be rolled out.
Our main goal right now is to provide a wide range of benchmarks so that if one area might be weaker than others, another testing segment can be more relevant for a given workload. Its really coming down to what the best stuff we have on hand at a given time during a review. Some of the behind the scenes stuff is growing the lab and improving some of the same tests that are in need of an overhaul. So you can be assured that we are taking note and looking at all new ways of testing

.