Sign in to follow this  
Brian

EMC Claims SSDs Will Overtake Tape for Archiving

Recommended Posts

I was just reading a report on Enterprise Storage Forum that pulled highlights from the upcoming (May 4th) release of the EMC-IDG Digital Universe study. Much of the commentary comes from EMC's Vice President and Global Marketing CTO Chuck Hollis. Hollis has some pretty standard takes throughout, but at the end of the article comes this little nugget on SSDs:

Hollis also sees a bright future for solid state drives — if they can follow the same evolutionary path as other silicon.

If flash follows the CPU market and sees the kind of performance and capacity growth and price reduction as Intel chips have seen, then flash drives could eventually become cheaper than hard drives and replace them — and perhaps even replace tape for archiving.

"It could get real interesting in four to five years," he said.

I know the pace of technology is fast, but does anyone believe that capacity, cost and reliability will magically align in 4-5 years so that SSDs can replace tape for archiving? I can't tell if Hollis was taken out of context here or if he's full of marketing-babble-BS.

SSDs need to catch HDDs first, and that seems unlikely in 4-5 years...forget about catching high capacity tape.

Share this post


Link to post
Share on other sites

I was just reading a report on Enterprise Storage Forum that pulled highlights from the upcoming (May 4th) release of the EMC-IDG Digital Universe study. Much of the commentary comes from EMC's Vice President and Global Marketing CTO Chuck Hollis. Hollis has some pretty standard takes throughout, but at the end of the article comes this little nugget on SSDs:

I know the pace of technology is fast, but does anyone believe that capacity, cost and reliability will magically align in 4-5 years so that SSDs can replace tape for archiving? I can't tell if Hollis was taken out of context here or if he's full of marketing-babble-BS.

SSDs need to catch HDDs first, and that seems unlikely in 4-5 years...forget about catching high capacity tape.

Man... I really get sick of these big companies doing stupid announcements like this.

Tape backup or rather backup itself (also use Disk, Diskstage and Dedup) is my daily job. Tape technology will be on LTO7 in 4 to 5 years. This is per normal roadmap. LTO5 was released about a month ago and is shipping currently, and new LTO tech gets released every 2 to 2,5 years. This means that a tape costing about 40$ a piece will hold roughly 6.4TB natively. Adding 2:1 compression this comes down to about 12TB. Normally I'd calculate with 1,5:1 compression, but with LTO6 they will introduce larger compression buffers which should allow for a bit better compression ratio (not the 2,5:1 they envision).

So, this guy could also say that in 4 to 5 years we will have SSD's disks of 12TB which will cost you only about 40$. Now that would be awesome! :D ... sadly... I think that to be complete fiction.... and not even close to what the truth is going to be.

Sure, SSD's are great now and will only become better. They will push forward tech like 6G SAS and SATA as mechanical HDD's are not pushing these limits and won't be soon.

Personally, I think 15K and partly 10K spindle disks will bite the dust sooner. You will get hybrid arrays which use 2TB Data disks (5400rpm or 7200rpm) with a 2 stage cache in front of it. Let's say 4GB memory cache per controller and 1TB of SSD cache as a sort of L2 cache. ZFS already can function in such a way.

So, going with EMC standards what I think we'll have in a few years:

2x8GB memory cache (dual controller)

10TB SSD L2 cache

250TB SATA 7200 RPM

Writes will go through SSD and cached read will also. This means your basically working on SSD but in the background it gets flushed to HDD. Actions can be streamlined (sort of advanced form of NCQ) and viola, the perfect, super fast, low response time solution.

Doing Backup and Storage and also a bit of archiving. In almost all companies, looking at their array's only a fraction of their online data is actually used. So for a 250TB array, 10TB can hold almost all the data anyone in the company is working on. A lot of data gets stored and files grow bigger, but what is really being worked on.... doesn't change that much in size. Also, if you do this using a disk array. This technique can be applied block wise. A whole file does not need to be cached, just the blocks frequently used or being written. 7200RPM aren't slow, just not that good at seeking, so that needs to be cached.

Anyway, I'm rambling now. Back to topic. I believe hybrid storage is the future. Dedup is getting important in the backup world, disk backups are fading fast and either used as a temporary stage for tape or being replaced for dedup storage. Tape is not dead, not by a longshot, neither SSD, Disk or Dedup hold the same advantages (yes, it also has disadvantages).

Edited by Quindor

Share this post


Link to post
Share on other sites

Quindor, you said it all. Let me just add a tiny bit:

If flash follows the CPU market and sees the kind of performance and capacity growth and price reduction as Intel chips have seen

That's soooo stupid! Intel and the rest of the microprocessor industry started their run about 40 years ago with feature sizes of several microns. Today we're at feature sizes of 32 nm (even shorter gate length). 1 nm is just 2 unit cells, or 4 atom layers. Flash absolutely needed these small features to even start to be interesting.. now there's much less room for growth, process-wise. It should still be possible to make it cheaper, even at stagnating cell sizes.. but that's not what Moores Law and the reference to Intel and CPUs is about.

MrS

Share this post


Link to post
Share on other sites

I can see SSD being used for snap backups, but i have a hard time believing that it will replace taps. that being said, one technology out today that has the possibility to replace tapes is Holographic Versatile Disc (HVD) with up to 5TB storage at the moment and 10TB expected. The main problem is that it is a write once media.

Side note: I think that Holographic Versatile Disc (HVD) will be the real successor to Blu-ray HD content, not some cloud storage.

Share this post


Link to post
Share on other sites

Side note: I think that Holographic Versatile Disc (HVD) will be the real successor to Blu-ray HD content, not some cloud storage.

Bah, physical media is dead...it just doesn't know it yet. Content delivery is already trending online/digital delivery, I think that will continue, albeit perhaps not at a rapidly accelerating pace. I still know people with VHS players for crying out loud.

Share this post


Link to post
Share on other sites

I am disagree with the time scale not that it may not eventually happen. The first fundamental issues with "cloud" distribution. In order for the internet to become the dominate distribution method it has address a few things.

BR currently averages 17-24Mbps and tops out at 50Mbps, faster then most home connections save for some variants of Fiber optic. DVD averages around 1.5Mbps but can spike up to 9Mbps. This also does not account for latency either. which can also be an issue. So this rules out the on demand ability for the time being. yes, i could "archive" it to another media such as HD, but then what is the fundamental difference between media and and HD other then HD are less portable?

Second one is trickier in the aspect that there is no solution other then time and that is people tend to have a need to have the textural object. That's one of the reasons why print media is still around. Ironically enough the biggest boost for print media was the computer revolution. Over time the younger generation will not be so enamored with having the physical media, but that's going to take decades

The final issue is CD/DVD/BR/HVD are not susceptible to EMP radiation and make a much better backup media because of it. That way I think that HVD will viable replacement for the majority of tapes.

BTW DVD to BR adoption is still faster the VHS to DVD ever was.

Edited by Mkruer

Share this post


Link to post
Share on other sites

Interesting discussion. Let's throw in some more bits:

BTW DVD to BR adoption is still faster the VHS to DVD ever was.

I think that's because VHC -> DVD was a disruptive change, whereas DVD -> BR is evolutionary in the way that your BR player can still deal with DVDs, so upon making the transition you loose nothing except money.

The final issue is CD/DVD/BR/HVD are not susceptible to EMP radiation and make a much better backup media because of it.

Good point. Longevity is also working against flash: here you are storing a few 100 electrons and need them to stay there, otherwise your data is lost. It works amazingly well, but is inherently limited and only becomes worse as more data is stored in each cell / transistor and the dimensions are reduced. You could refresh it every now and then, but that requires quite some effort (read & re-write each and every single bit) and is not what you want for long term archives.

And online content delivery is not there yet for full scale adoption. We'd need serious infrastructure / bandwidth enhancements. Recently I was watching a free movie from Maxdome.de.. the experience was quite horrible, nothing like running a local file. It was more a problem of software stability rather than bandwidth, though, so this could be fixed. In the medium to long term I see one major argument for online content delivery: digital rights management is much easier to implement. People need accounts, can be traced and can't rent or resell stuff. These might not seem very compelling to you, but it does to *someone else* ;)

MrS

Share this post


Link to post
Share on other sites

I'd concur with physical media not going away any time soon. They may be damaged under some circumstances but won't magically disappear (remember that embarrassing Kindle incident, involving an Orwell e-book of all things?), and like stated have some intrinsic value. Besides, it is easier to import a CD from another country than the same for the electronic version, ironically enough... (This reason for this is quite simple: a much more decentralized distribution network. Things electronic tend to have a "gatekeeper problem".)

Share this post


Link to post
Share on other sites

There is another technical reason why I think HVD will be the last of the disc movie formats. I think by the time that HVD makes it to the scene; the internet will be just starting to have the bandwidth for the masses in order to play current HD (50Mpbs) content on demand. But the real reason is far simpler. Simply put once you go to a 4k image, you will not get any additional (relevant) information off the older 35mm film so for all intents and purposes cinema 4k is the end of the road in terms of converting 35mm to digital. This jump right now would require approximately 2 to 4 times the current space or about 100 to 200 GB, but you need to put on top of that h.264 (mpeg4/avc) level 5.1 and you go from 50Mbps to 250Mbps or 5 times more bandwidth for a more accurate representation of the original 35mm film and because h.264 is still evolving we will probably see additional level including lossless encoding.

Cinema 4K = 4096x2304 (16:9) Each uncompressed frame is 4096*2304*(14+14+14) height width (Y'CbCr) or 47.25MB per frame. Assume that the average move is 2hrs long, and you get 174,800 frames

A movie fully uncompressed is about 7.8TB worth of raw data.

Using the current AVC compression the average space required is ~3.5% of the raw size, assuming the same settings.

That means that if BluRay wanted to fit a full movie using Cinema 4K it would require a 250GB disc.

Does anyone need “more quality” well going from DVD to BR was ~10x- improvement in overall quality.

A jump from BR to HVD would be at minimum a 5x improvement but add fully uncompressed video, and the overall quality should be on par with the DVD to BR jump if not greater.

Share this post


Link to post
Share on other sites

SSDs have physical advantages over tape and rotating disks for backup.

My company currently backs up to LTO3 and has significant shoe shining occurring. I'll be switching to LTO4 very soon and will be testing turning off compression to reduce shoe shining.

We also have an accounting employee carry the tapes to the bank for secure off site storage. I wouldn't trust hard drives to survive the beating they'd probably take going in and out of the lox box at the bank. SSDs would also weigh less than LTO or hard drives (significant when the accounting employee complains about the weight of a locked briefcase for transporting backup media).

SSDs are random write so they don't shoe shine like LTO

SSDs are practically immune to shock damage so they aren't vulnerable like hard drives.

It'd take a big drop in cost that hasn't happened yet but if I could get SSDs at even twice the cost per GB of LTO4 media it'd be significant considering a Quantum Superloader 3 is about $4000 and eSATA cradles are less than 1% of that cost. Even versus a non robot(tape loader) LTO drive the comparison is still several thousand dollars vs almost nothing for a cradle or enclosure.

For really small businesses backing up to SATA SSD (by way of eSATA cradle or USB3 superspeed enclosures) will be cost effective well before the per GB cost of the raw media equalizes in price.

Edited by dhanson865

Share this post


Link to post
Share on other sites

EMC is full of %@$#.

LTO will be hitting its last generation (LTO6) in about 3 years. In 4 or 5 years very fast perpendicularly-recorded 50 TB tape will probably be available. Super LTO anyone???

Share this post


Link to post
Share on other sites

but if I could get SSDs at even twice the cost per GB of LTO4 media it'd be significant

An interesting benchmark. Let's put in some numbers:

LTO4 is at about 25€/tape (could be as low as 23 from what I see upon a quick google search). It offers 800 GB uncompressed capacity and I figure it's only fair to compare this to SSD capacity, since you could compress them just as well. That's 0.03 €/GB. If the SSD has to reach twice that it'll be 0.0625 €/GB.

Currently we can get 64 GB for 150€. Let's assume that for backup purposes all of this is user accessible. That yields 2.3 €/GB, i.e. a factor 37.5 too large. A full die shrink usually offers a sqrt(2) reduction in feature length and thus sqrt(2)^2 = 2 times the transistor density and we can thus assume cost/capacity to drop to 1/2 upon each die shrink. That'll mean we need 19 of them to reach your target point. If all goes well we can get a die shrink every 2 years, so this will take us 38 years. If all goes well. And by the time we get there we'll be talking about 34 nm / (sqrt(2)^19) = 0.05 nm technology. Remind you that the spacing between 2 atoms in Silicon is 0.25 nm, so I don't think Flash is going to make it ;)

You could of course argue that they could get away with triple bit cells for backup and that additional cost saving due to economy of scale might kick in. But you can also argue that we're comparing to a static LTO4 benchmark here, which is probably not completely realistic either.

MrS

Share this post


Link to post
Share on other sites
MrSpadge' date='04 July 2010 - 05:47 AM' timestamp='1278236852' post='262532']

An interesting benchmark. Let's put in some numbers:

LTO4 is at about 25€/tape (could be as low as 23 from what I see upon a quick google search). It offers 800 GB uncompressed capacity and I figure it's only fair to compare this to SSD capacity, since you could compress them just as well. That's 0.03 €/GB. If the SSD has to reach twice that it'll be 0.0625 €/GB.

Currently we can get 64 GB for 150€. Let's assume that for backup purposes all of this is user accessible. That yields 2.3 €/GB, i.e. a factor 37.5 too large. A full die shrink usually offers a sqrt(2) reduction in feature length and thus sqrt(2)^2 = 2 times the transistor density and we can thus assume cost/capacity to drop to 1/2 upon each die shrink. That'll mean we need 19 of them to reach your target point. If all goes well we can get a die shrink every 2 years, so this will take us 38 years. If all goes well. And by the time we get there we'll be talking about 34 nm / (sqrt(2)^19) = 0.05 nm technology. Remind you that the spacing between 2 atoms in Silicon is 0.25 nm, so I don't think Flash is going to make it ;)

You could of course argue that they could get away with triple bit cells for backup and that additional cost saving due to economy of scale might kick in. But you can also argue that we're comparing to a static LTO4 benchmark here, which is probably not completely realistic either.

MrS

So you think in € and I think in $. Either way my statement of twice the price per GB versus LTO4 includes the cost of the drive not just a blank media. If you have a $3000 drive and you archive two tapes a month for a 3 year period then the cost of using LTO4 isn't $0.05/GB it is $0.10/GB. If you only archive 1 tape per year the cost per GB is $1.3 about 25 times higher. If you archive less tapes per year the price per GB goes up, if you archive more tapes per year the price per GB goes down.

So for the small shop, casual user, and what not LTO is too expensive as they can just buy $70 hard drives and get $0.10/GB or better at much lower initial costs.

For those types of users SSDs only have to become less than twice the cost of a hard drive or somewhere under $0.50/GB and they'll stop using hard drives for off site storage. How many years do you think that will take?

For users that use dozens of LTO 4, LTO 5, LTO 6 per year (yearly archives, quarterly archives, monthly archives, weekly archives) yeah the LTO tapes will be half the cost of hard drives and then you can work out the math between hard drives and SSDs.

While there is a wall coming for reducing feature sizes .25nm isn't the way they name the node size wall. 22nm processes are coming online very soon. Additional cost savings can come by increasing wafer diameters or creating larger factories or more factories it doesn't have to be all about new process dimensions.

While we are on the topic of predicting the future lets talk unit share.

Right now there is a price premium on SSDs due to scarcity. If you could somehow make every hard drive on the planet disappear instantly there wouldn't be enough SSDs to replace them.

Flat out, not enough SSDs on the planet to replace all the hard drives on the planet.

Until that fact changes there will be a price premium on SSDs. Now in the enterprise space there are uses where one SSD can replace 10 hard drives. So we don't have to have 1 SSD for every hard drive. At home you may need 1 SSD to replace two hard drives if you are the type to do RAID 0 with a couple of raptors. The vast majority of the world though will need 1 SSD for every hard drive replaced as they don't use more than one hard drive in their Desktop, laptop, or other device.

So lets say on the grand scale we need 1 SSD for every 1.5 hard drives we want to retire.

2010 unit shipment forecast for hard drives is 674.6 million drives (up over 20% from 2009 if the estimate is accurate). The first number I found for SSDs says 11 million drives in 2009 (with expectations of 90% growth in units).

As SSD supplies increase their price premium will drop. Eventually the units shipped for hard drives will decrease instead of increase. Eventually SSDs will see growth rates higher than 100% year over year as a massive transition occurs. But for now lets imagine what it's like if HD units decline by 10% a year while SSD units double per year. Where does that line cross? It takes about 6 years at that rate. How about if HD units drop by 10% a year while SSD rates triple each year, in that scenario it takes about 3 years to even up on the per year unit rate.

Articles like http://www.fabtech.org/editor_s_blog/_a/wheres_the_fab_capacity_needed_in_2010/ discuss a shortage of wafers in the industry which will artificially constrain SSD growth rates until 2011 and I am no soothsayer. I don't know how quickly SSDs will ramp. I do expect them to eat into HDs in the near future and I do expect when they become reasonably priced people will stop backing up to hard drives and instead backup to SSD.

Share this post


Link to post
Share on other sites

My company currently backs up to LTO3 and has significant shoe shining occurring. I'll be switching to LTO4 very soon and will be testing turning off compression to reduce shoe shining.

This is a backup software / underpowered hardware issue. The problem is caused by poor or poorly configured backup software not buffering the data before it is sent to the tape drive and also hardware that can not keep up with 120MB/s streaming. Both are easily solvable. A modern quad core server with 4+ GB of ram and a raid0 of ssds or velociraptors ... can go a long way for the hardware. Make sure the raid is only used for the buffer and not used at all for the source data. For the software I use bacula which supports buffering the data before it goes to the tapes. Backups spool to a spool area and that gets despooled at tape drive speeds.

Edited by drescherjm

Share this post


Link to post
Share on other sites

Man... I really get sick of these big companies doing stupid announcements like this.

Tape backup or rather backup itself (also use Disk, Diskstage and Dedup) is my daily job. Tape technology will be on LTO7 in 4 to 5 years. This is per normal roadmap. LTO5 was released about a month ago and is shipping currently, and new LTO tech gets released every 2 to 2,5 years. This means that a tape costing about 40$ a piece will hold roughly 6.4TB natively. Adding 2:1 compression this comes down to about 12TB. Normally I'd calculate with 1,5:1 compression, but with LTO6 they will introduce larger compression buffers which should allow for a bit better compression ratio (not the 2,5:1 they envision).

So, this guy could also say that in 4 to 5 years we will have SSD's disks of 12TB which will cost you only about 40$. Now that would be awesome! :D ... sadly... I think that to be complete fiction.... and not even close to what the truth is going to be.

I agree this sounds absolutely ridiculous. Even counting the price of the tape drive in the calculations. SSDs are way too expensive per gigabyte. Also I thought that SSDs had issues with data retention when not powered up for 2 or so years.

Share this post


Link to post
Share on other sites
MrSpadge' date='04 July 2010 - 02:47 AM' timestamp='1278236852' post='262532']

An interesting benchmark. Let's put in some numbers:

LTO4 is at about 25€/tape (could be as low as 23 from what I see upon a quick google search). It offers 800 GB uncompressed capacity and I figure it's only fair to compare this to SSD capacity, since you could compress them just as well. That's 0.03 €/GB. If the SSD has to reach twice that it'll be 0.0625 €/GB.

Currently we can get 64 GB for 150€. Let's assume that for backup purposes all of this is user accessible. That yields 2.3 €/GB, i.e. a factor 37.5 too large. A full die shrink usually offers a sqrt(2) reduction in feature length and thus sqrt(2)^2 = 2 times the transistor density and we can thus assume cost/capacity to drop to 1/2 upon each die shrink. That'll mean we need 19 of them to reach your target point. If all goes well we can get a die shrink every 2 years, so this will take us 38 years. If all goes well. And by the time we get there we'll be talking about 34 nm / (sqrt(2)^19) = 0.05 nm technology. Remind you that the spacing between 2 atoms in Silicon is 0.25 nm, so I don't think Flash is going to make it ;)

You could of course argue that they could get away with triple bit cells for backup and that additional cost saving due to economy of scale might kick in. But you can also argue that we're comparing to a static LTO4 benchmark here, which is probably not completely realistic either.

MrS

:-) Minor math flaw there - to paraphrase, you're saying "it will take 19 rounds of cutting the $/GB in half to reduce it by a factor of 37.5". Nineteen cutting-in-halfs is 0.5^19 = around 2/1,000,000.

You'd need about 5 rounds of cutting the $/GB in half (2^5=32).

That still means 10 years and about a 6um process using your other estimates / extrapolations - not really something near-term, still, so your argument still holds water.

Share this post


Link to post
Share on other sites

Oh dear.. where was my mind?! You're absolutely right :o

@dhanson865: you're right, there can be additional cost saving mechanisms for SSDs. However, the larger wafers actually seem troublesome: 10 years ago we were meant to have moved from 300 to 450 mm wafers since a long time by now. Never happened, though. Either the crystal growth guys can't do it (at electronics grade quality) or there are other problems further down along the road (I wouldn't think that anything here would actually be prohibitive, but the devil's probably in the detail). And for processor manufacturing you benefit a lot from maturing yields over time. This should be much less of a factor for flash, since the entire structure is pretty much redundant: let a bit fail at manufacturing or later one - you don't care, just use one from the spare area.

And on the other hand die shrinks are becoming ever more difficult: they're confident for 22 nm and a little below that, but we're still doing it with 193 nm light sources. Good Mr Abbe would be amazed at how we've been tricking our way around his fundamental limit. We don't have a viable light source yet to go any further once we run out of lithography tricks. Not saying it can't be done, just that there'll be plenty of trouble down the road ;)

And switching from HDDs to SSDs for "small" backups. That's an interesting thought, although intuitively I'd strongly reject it. If capacity is enough and mechanical handling of the HDDs is a problem I could see them move to 2.5" HDDs first. Right now they're at 500GB for 50€ or 640GB for 55€ and they can take quite a beating (dropping your notebook etc).

And generally to me it just feels conceptually wrong to try to backup on SSDs when other methods are readily available. Their advantage is speed, but every single bit costs you just as much as the previous one. For HDDs on the other hand it doesn't matter much if it's got 1, 2 or 4 platters (or 5 if you're Hitachi). And at the 50 nm node each bit in a single level cells is stored by just a few hundred electrons (I think I heard ~500). This is only getting worse as dimensions are reduced. Sure, you can calculate the average escape rate and run a refresh every now and then.. but still, conceptually it doesn't feel right. Especially since it's not like we wouldn't have other methods to choose from.

I dare say: before we see widespread adoption of flash for backups we'll see HDDs specialize on backups. Relieved from the burden to provide anything else than mediocre STR (USB limit) one could reduce the spindle speed below 5400 rpm again (if this leads to significant capacity increases). 5 platters in 3.5" are welcome for such a task and are easier to implement for slower spinning drives. And then we could also go back to 5.25" drives! Mind you, only for backups.. they'd be rediculously slow as system drives. Here one would have to use something in the range of 3k - 4k rpms, as otherwise power consumption and platter fluttering would become problems. But just imagine how much storage you could put on there: 8+ platters of huge diameter. Add still moderate power consumption (<20W) if powered up for the backup and an STR > 100MB/s especially in the outer regions.

Economically that would make so much more sense. Equip PCs with SSDs (if they don't work on large datasets) or hybrids / dual drive setups (SSD+HDD) if they've got to do some heavy lifting. And do the backups centralized on specialized machines automatically over LANs. Or at home equip one machine with e.g. 2 1.5 TB 5400 rom drives in Raid 1 and have all other ones use these for backup. It's got to be as easy to set up as e.g. the backup built into Win 7, though.

Edit: Behemoth HDDs for backup and media servers, of course.

MrS

Edited by [ETA]MrSpadge

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this