tealeaf

Does mechanical storage have a future?

Recommended Posts

Not to mention that the ki vs k issue was just ignored because the difference was just 2%... but with Mbyte, it was already nearly 5% , and with TB it will be >10%, making it unignorable.

The computer world IS a mess in its incoherent missuse of SI units. Your computer running at 1Ghz doesnt run at 2^30 Hz, and nobody cries "I got cheated by 75 Mhz!!".

All transfer rates that are time based (be it network, modem, interconnects and busses) also are 10^x based.

The only thing in binary are solid-state memory and the way OSes show HD-space.

If they just finally changed the default behaviour of showing hd-space, bitching would suddendly drop by 90%.

Share this post


Link to post
Share on other sites
All transfer rates that are time based (be it network, modem, interconnects and busses) also are 10^x based.

But there's no reason for them to be.

The only thing in binary are solid-state memory and the way OSes show HD-space.

If they just finally changed the default behaviour of showing hd-space, bitching would suddendly drop by 90%.

Isn't a sector 2^9 bytes?

And most other apps use binary units too.

Share this post


Link to post
Share on other sites

"But there's no reason for them to be."

And there's no reason for them to be binary either.

"And most other apps use binary units too."

But some applications have started using KiB, MiB and GiB. And yes, they are still binary units! Maybe all the rest would use them too if Microsoft would cease using KB, MB and GB and use the correct abbreviations.

It's perfectly OK for me that there's binary and decimal kilobytes, as long as they are abbreviated differently (KiB for binary and kB for decimal). Notice that when using SI prefixes, the k is not a capital letter. (Maybe the "KB" has a capital K because old operating systems used only capital letters... We really should get rid of that capital K and either substitute it with k or Ki, depending whether we mean binary or decimal.)

Share this post


Link to post
Share on other sites

And there's no reason for them to be binary either.

The base unit of memory is binary. The base unit of storage is the sector (often 512 or 4096 bytes), which is also binary.

What do you mean there's no reason?

But frequency is NOT binary. So why should your 100Mhz clocked bus driving 64 bit per clock now be 62.131 MiB/s instead of 64MB/s?

A Megahertz HAS 10^6 cyclees per sconds...

Share this post


Link to post
Share on other sites

Back onto the topic of SS-memory being slow, putting 20x16GB chips into a 3.5" form factor to achieve 320GB can also (theoretically) give you 20x the throughput, easily reaching over 100MB/sec. That is, if you have the control circuitry to go along with it, which wouldn't be that hard given today's technology.

In other words, stick 20 16GB flash cards in RAID and you won't be complaining about the speed anymore - especially not since solid-state also has the potential of having much lower access times than hard drives.

Share this post


Link to post
Share on other sites
But frequency is NOT binary. So why should your 100Mhz clocked bus driving 64 bit per clock now be 62.131 MiB/s instead of 64MB/s?

A Megahertz HAS 10^6 cyclees per sconds...

It'd vote for 8 megabyte/s. But for frequency the base is indeed arbitrary.

Share this post


Link to post
Share on other sites
Back onto the topic of SS-memory being slow, putting 20x16GB chips into a 3.5" form factor to achieve 320GB can also (theoretically) give you 20x the throughput, easily reaching over 100MB/sec. That is, if you have the control circuitry to go along with it, which wouldn't be that hard given today's technology.

In other words, stick 20 16GB flash cards in RAID and you won't be complaining about the speed anymore - especially not since solid-state also has the potential of having much lower access times than hard drives.

Well, today's flash memory is structured in 512 byte sectors (to mimic hard drive organization) so putting 20 together in a raid array would require you to write in units of 10K disk blocks for best efficiency. To keep things compatible with existing device drivers, you would need to use powers of two, and smaller page sizes on the individual flash chips. Another possibility is migrating to a larger minimum block size, there's talk of switching to 4K sectors for disks. With 4K sectors you could run 8 flash chips in a parallel RAID without any difficulty, and 8x throughput would be fast enough to max out SATA1. Or go with 16 in parallel at 256 bytes per sector, that'd max out SATA2.

Share this post


Link to post
Share on other sites

Would you need to manufacture flash memory cells differently if it was going to be used in parallel, or would it just involve a change in the controller logic? I don't get why someone can't produce a single device with 8 times the usual number of flash memory chips, a controller to run them in parallel for 8x the throughput and still present 512 byte sectors to the outside world by 'invisibly' RAIDing internally, writing 64 bytes to each chip.

How is sector size a physical property of the flash memory? How hard is it to modify that property to allow the above to work?

Share this post


Link to post
Share on other sites
Would you need to manufacture flash memory cells differently if it was going to be used in parallel, or would it just involve a change in the controller logic? I don't get why someone can't produce a single device with 8 times the usual number of flash memory chips, a controller to run them in parallel for 8x the throughput and still present 512 byte sectors to the outside world by 'invisibly' RAIDing internally, writing 64 bytes to each chip.

How is sector size a physical property of the flash memory? How hard is it to modify that property to allow the above to work?

Check out the data sheets for one of these regular NAND parts, e.g. Samsung 1Gbit NAND

You'll see that the sector size is very much a physical property of the memory array. Of course, looking at the block diagram, you see that the memory array is quite separate from the controller. Perhaps they should just design a controller with 8 times as many I/O lines, and widen the arrays by a factor of 8, etc., to increase parallelism. That would probably drive the per-chip cost up tremendously - otherwise why would they still be multiplexing address and data lines everywhere?

Share this post


Link to post
Share on other sites

Admittedly, you would indeed have to write in blocks of 10K or so for the best efficiency, but then again, 512b blocks aren't exactly efficient for current hard drives anyway - they're optimal at block sizes of about 32/64K AFAIK. Finally, 512 byte blocks aren't the most used, nor is reading or writing a single 512 byte sector going to take a particularly long time, whatever the sector size. Therefore, I don't think it's going to be too big of a problem.

Running chips in parrelel cound in theory be accomplished simply by a change in controller logic and/or design - similar to a RAID array uses standard hard drives but an improved controller. This kind of implementation would share similar properities to a RAID array - notably a stripe size and an optimal block size as a result. There's no need to "invisibly RAID" a single 512 byte sector into 64bytes per chip. You could just leave it at 512 bytes and only "RAID" them when the read/write block size exceeds a single sector - e.g. a 4KB read being RAIDed across 8 chips, a 16KB read being RAIDed across 32 (assuming you have that many).

Now, I'm not entirely certain about the access protocols and features of flash-type memory but I'm largely confident that this would work. Admittedly, there will be an increase in per-megabyte or per-gigabyte cost as a result of extra controller logic, but it wasn't too long ago that RAID controllers cost upwards of $300 for a basic one, now they're being included on budget motherboards for $5 a piece or less.

I suspect that a similar type of parralel "RAID"ing may be what Sandisk is using to speed up their Ultra II and Extreme III cards - it seems logical that it would probably be cheaper (somewhat) to improve controller logic to run chips in parralel than to develop a new, faster chip. But again, I'm no authority on the matter so it's just speculation on my part.

Share this post


Link to post
Share on other sites
I suspect that a similar type of parralel "RAID"ing may be what Sandisk is using to speed up their Ultra II and Extreme III cards - it seems logical that it would probably be cheaper (somewhat) to improve controller logic to run chips in parralel than to develop a new, faster chip. But again, I'm no authority on the matter so it's just speculation on my part.

I know of a few companies making dual channel USB flash drives, these are obviously writing to two flash chips in parallel to get higher speed. I expect this will become more and more common over time.

Share this post


Link to post
Share on other sites

In response to the original question, I think the answer, at least in the short term, is "yes".

Just as 5.25" full height drives gave way to 3.5", the drives in the near future will be the smaller 2.5" (and smaller) drives. In a few years we'll be looking at our old 3.5" beasts and thinking "when will this dinosaur die so I can go get a new 1.25 TB 2.5" 7200rps drive?"

....mechanical storage won't die anytime soon, it will simply get faster, smaller, and higher capacity than the drives before.

I sure hope I'll be able to plug my closet full of offline/archive storage drives into a computer for at least a few more years!

my 2cents.

Spinme

Share this post


Link to post
Share on other sites
In a few years we'll be looking at our old 3.5" beasts and thinking "when will this dinosaur die so I can go get a new 1.25 TB 2.5" 7200rps drive?"

I hope it'll be a 2.5" 10000rpm drive :)

Derek

Share this post


Link to post
Share on other sites

spinme: "....mechanical storage won't die anytime soon, it will simply get faster, smaller, and higher capacity than the drives before."

I don't think that small HDD have a long future. It's simply because they contain the same amount of moving part as their big cousins but offer significantly less capacity, while having to compete with the performance, reliability, power consumption, shock tolerance and cost of the solid state memory. Consider 1" microdrive... do you expect it to prevail as long as 2.5" and 3.5" HDDs? I wouldn't. 8GB storage on a platter with less than 1" diameter and a very small actuator seems pretty non-cost efficient solution for the time to come, considering there's already flash based solution of several gigabytes. It won't take long before there's cheap 10GB flash media. By that time, there would probably be microdrives with about 20GB capacity using perpendicular recording but most likely with a price tag not too emphasasing... not to mention the low MTBF value and short warranty of smaller form factor HDDs, while flash media is offered with 5-year-warranties. Even Seagate doesn't apply it's 5-year-warranty scheme on their pocket hard drives.

I think 2.5" and 3.5" form factors will be the last fortresses of mechanical storage. When the vibration of internal components prevents use of higher density in "3.5 inch" form factor (which use approximately 4 inch platters), I'd consider that to be a sign that the reign of mechanical storage will soon come to it's end. If HDDs will transition to 2.5 inch for factor, the solid state memory will catch mechanical memory faster. That is simply because 2.5" drives use platters of half the diameter and thus one quarter the area. Areal capacity have to be quadrupled to obtain the same capacity. Also there's a limit on how many platters you can fit into slim 2.5" form factor. Usually they contain 2 platters, where as "low-profile" 3.5" has up to 5 platters (Hitachi) or 4 platters (other manufacturers, except Samsung with just 3 platters). Since the number of platters that can be used is halfed, the areal density have to be hextupled, octupled or decupled to obtain the same capacity.

I don't see they'd abandon 3.5" format any sooner than absolutely necessary. But I do foresee people building silent computers and "home-theater clients" using a single local 2.5" HDD for OS and programs and possibly a separate file server (possibly a NAS containing several terabytes of storage).

Edited by whiic

Share this post


Link to post
Share on other sites
I don't see they'd abandon 3.5" format any sooner than absolutely necessary. But I do foresee people building silent computers and "home-theater clients" using a single local 2.5" HDD for OS and programs and possibly a separate file server (possibly a NAS containing several terabytes of storage).

I'd hope MS properly supports network booting instead.

Share this post


Link to post
Share on other sites

I don't know that vibration will necessarily be the death of hard drives. I'm sure there are bearings significantly better than what the hard drive industry currently uses. I would hazard a guess that there are industries that require extremely precise bearings (moreso than hard drives, I just doubt that the bleeding edge bearing technology is being used in consumer hard drives).

Just my judgement and it depends how long we are talking. Will hard drives be around for ten years? IMO absolutely. 50 years? IMO most likely not. People talked about the end of hard drives in the late 80s because head and bearing technology was not sufficient. New tech and fixes to barriers are always found if it's economical. The end to it being economical will be a cheaper, faster and more dense technology. If no such technology exists, hard drives will continue to reign. At this point we can only guess. Hard drives have been in PCs for 25 years and unless something BETTER (better enough to justify changing new PCs to the new tech) comes along, again, HDDs will remain inside PCs. In other words, if a technology too similar to hard drives (in speed, density or price) comes along, it probably won't change what PCs come with (it will give HDDs a run for their money of course).

Just my ramblings, nothing can be said for certain.

Share this post


Link to post
Share on other sites

I think Whiic is on the right track. These guys already spelled it out in their chart of System cost vs # of MB - even back in 2004 flash memory was more cost effective for small storage devices. 1: there is a trend towards more and more use of smaller form factor drives (pocket drives and microdrives) and 2: flash memory has already exceeded HDD's price/performance here due to the fixed overhead cost of HDD mechanisms.

Edited by hyc

Share this post


Link to post
Share on other sites

hyc: "I think Whiic is on the right track."

But I did make at least one grave mistake when calculating areal densities. I assumed 2.5" to have platter with half the diameter (and thus surface are of 1/4). That isn't true. They have platter size that is half the surface area of 3.5" drives.

Because of this "hextupled, octupled or decupled" should be "tripled, quadrupled or pentupled" (alternative spelling of pentuple is quintuple).

Share this post


Link to post
Share on other sites
If one compares the physical dimensions of a CompactFlash card with a 3.5" ATA drive and then calculate how many CompactFlash cards would fit into the same physical space and also take into consideration that CompactFlash cards are now available in 16 GB sizes one is bound to question the future of mechanical mass storage.

Given that the current $/GB sweet spot of a magnetic HDD is about 1/100th that of high-performance CF cards, it will be a while before it no longer makes economic sense to use HDD's. I don't know of any retailer that is selling (or even advertising) 16GB CF cards. Even the Seagate 750GB HDD is in better supply. :lol: 8GB CF cards are as large as you will commonly see today.

I'm adding another 2 TB to my workstation at home. If I want it to be fast with random read/write access, and affordable by mere mortals, the HDD is currently the only solution around, with nothing else in sight. Eventually, yeah, spinning magnetic platters might fall by the wayside, but that won't happen anytime soon.

Share this post


Link to post
Share on other sites

I agree with others that flash memory appears likely to catch up to HD in a price/GB in several years time for the 3.5" size. And although I too originally thought transfer speed will be a problem, it's true you could use some sort of array (I avoid use of the term RAID because what we're talking about is RAID 0 which of course isn't really RAID since it isn't redundant). The complexity of such a design in the circuit level is an interest issue which I have no idea on but I'm guessing it will be possible.

However the bigger issue to me that hasn't really been discussed is the erase cycles issue. AFAIK (wiki), it's still at about 1 million. IMHO, this isn't really enough. You'd at least need some sort of allocation system to ensure the same sector isn't used excessively, at some level. This is going to add complexity somewhere along the line. According to wiki, some devices already do this. Does anyone know how well it's done? Also, is the 1 million cycles improving? Would it be 100 million or better in e.g. 5 years?

Share this post


Link to post
Share on other sites

It wasn't so very long ago that everyone thought that a 4.7 GB hard drive was more than one would ever need, wasn't it?

I don't think anyone ever thought 4.7GB hard drive is enough, but someone did think 640K of memory is enough for anybody :huh:

Really who? This statement is commonly attributed to Bill Gates but there is no evidence he ever said it and he denies it http://en.wikiquote.org/wiki/Bill Gates#Wrongly Attributed. It's possible someone at IBM said/thought it but I doubt it. It was simply a design decision based on the expected needs etc at the time and in the immediate future (good or bad one that's up to you to decide).

Edited by Nil Einne

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now