sync08

Home server with 50 drive

Recommended Posts

Hi all,

I'm considering to build a new home server which will hold 50x2TB drives. As a case, I'm considering 9U/50 bay chembro. But I still dont know how I'll able to connect more than sata drives that mainboard already supports? For example, I can connect 4 sata drives to my mainboard but for connecting more than 4 drives, what should I do? sata port multiplier? I really dont know how its being done, if someone can point me a way I'd be really appreciate it. .

Edited by sync08

Share this post


Link to post
Share on other sites

Well for starters its going to be an expensive project, so I have to ask... what is the intended goal? Are you creating a huge disk array for media storage? Is this for a home operated website? What is the budget?

For connecting a ton of drives you are going to need server hardware. This type of project you probably won't have more than one or two boot drives connected to the motherboard. The storage/array drives will be connected through RAID interfaces similar to this:

http://www.newegg.com/Product/Product.aspx?Item=N82E16816118089&cm_re=lsi_megaraid-_-16-118-089-_-Product

Share this post


Link to post
Share on other sites

A secondary SATA controller card would work, though you'd need several to get that many drives connected. That sounds like an awful lot of storage to pack into a small space, though. I'd worry about power consumption and cooling with that many drives in one case.

Share this post


Link to post
Share on other sites

First of all, thank you for your reply.

Well, my goal is to create a expandable system for my home storage which will last for 2-3 years.. I'm connected to 2Gpbs metro ethernet network and I have been authorized to leech it until to the last bytes :P

I'll be filling my drives with usuall stuff like mp3s,dvds, ebooks etc.. (usuall pirate stuff :rolleyes::ph34r: ) Plus I may be using it also for the Virtualbox..

About the budget, since its been sponsored by my boss we are all ok to take it to the limits :D

I have considered 9U Chembro as the case. It already has cooling fans inside the case but I never run 50bay server so I really have no idea what is the proper temp for it.. But I'm all open to any suggestions ..

1036chenbrorm91250backa.jpg

As PSU, I'm considering 1620 Watt 3+1 Redundant psu.

24GB DDR3 Ram

Mainboard - CPU -> Open for any suggestions

what else ? :)

I'm aiming to build a expandable system which will work without any issues and wont shutdown itself or create any issues like bluescreens..

Share this post


Link to post
Share on other sites

Dear Lord - that's pretty close to a media storage wet dream!

50 drives should last you a little bit...but aren't we being hasty, you know with 3 TB around the corner ;)

Did you decide on drives though? And you may consider a redundant SSD config for boot.

Incidentally - welcome to the forums and we have a home server dedicated forum. If this thread is more about the gear other than drives, we should probably relocate it.

Share this post


Link to post
Share on other sites

Also, how do you plan to back up all that data? Do you have a tape system or something else? What kind of performance do you need out of the array?

Share this post


Link to post
Share on other sites

Well, about 3TB drives , no need to be greedy but .. :lol::rolleyes:

SSD for boot, yes (But no idea which SSD, few of them seems to be having firmware issues, any decent one out yet?)

Regarding warm welcoming, thanks! and about relocation, I agree, sorry for opening the topic at wrong forum category :-/

Mickey,

No backup plans , but is there any possibility that I may need it because of the raid/multiplier setup? (I mean I already have 18TB fulled drives and none of failed or crashed yet)

My old setup: I was connecting my 2TB drives as secondary drive to my PC and download the content , make the disk full then disconnect and connect new 2TB.

Share this post


Link to post
Share on other sites

SSD for boot, yes (But no idea which SSD, few of them seems to be having firmware issues, any decent one out yet?)

In this case it's about reliability. Speed is nice, but you don't need to be blazing to get the job done. I'd strongly recommend the Intel X25M and be done with it. The smaller one would work...you could probably even run the Value 40GB unit if you wanted to save. In fact, I might RAID1 two of those smaller 40GB units and be happy.

We have two value drives and should be postings some RAID results soon.

Mickey,

No backup plans , but is there any possibility that I may need it because of the raid/multiplier setup? (I mean I already have 18TB fulled drives and none of failed or crashed yet)

My old setup: I was connecting my 2TB drives as secondary drive to my PC and download the content , make the disk full then disconnect and connect new 2TB.

I'm going to duck now as you're liable to get hit with a boot from any one of our regulars ;)

RAID isn't backup...and yes, you should have a plan unless you want to recreate everything. Forgetting about the box itself - fire anyone? Theft, etc, etc...do you really want to recreate all that content?

A3 could be a player here for automatic cloud backup.

Share this post


Link to post
Share on other sites

First of all, what OS do you intend to use? You mention "home server" but it isn't clear if you mean Windows Home Server or simply a home built server. I use WHS, and am quite pleased with it. If you do use WHS then you don't enable any sort of RAID as it performs file duplication for RAID1 at the file system level.

It also isn't clear what kind of backplane is used in that case. How many of what kind of connector is on the back? That will determine what kind of controller you'll want to get.

Share this post


Link to post
Share on other sites

Mickey,

No backup plans , but is there any possibility that I may need it because of the raid/multiplier setup? (I mean I already have 18TB fulled drives and none of failed or crashed yet)

I'm going to duck now as you're liable to get hit with a boot from any one of our regulars ;)

RAID isn't backup...and yes, you should have a plan unless you want to recreate everything. Forgetting about the box itself - fire anyone? Theft, etc, etc...do you really want to recreate all that content?

I prefer a more genteel approach than hurling footwear. :)

RAID is meant for uptime, not backup in the traditional sense. It provides no protection against fire, theft, a power short taking out your drives/system, virus, etc. And if most of your media is really from "alternative" sources, maybe you don't particularly care if you lose it. You could just collect it again. Or for stuff you ripped yourself, you could use the original media as your backup. I keep my CD collection around partly for that reason.

For backup, you need to honestly ask yourself how much of that data is critical to keep vs. good-to-have. The critical data I need would fit on one DVD (personal information, family pictures, etc.). The rest of it I already have on install disks or I can easily download again (web browser, email client, various utilities). And I keep a second copy, on an external HDD, of stuff recorded off my DVR, as it'd be a hassle to record it again. Still, that drive is only 500 GB total, with space to spare. It's not a practical option for 100TB of media.

Also, if you're going the RAID route, especially with that many drives, you have to carefully choose which particular flavor of RAID (and thus, what hardware) you get.

I'll let others with more experience chime in on how to pick a RAID setup, as it's not my area of expertise.

Share this post


Link to post
Share on other sites

I'd pick chassis with 6G SAS expander such as Supermicro. You only need one SAS port to the expander or 2 ports if you want multipathing. If you need to add more capacity, add another chassis and daisy chain them.

Also depending on what OS you use, some can do compression and deduplication which reduces total storage requirement.

Share this post


Link to post
Share on other sites

[snip]

I'll be filling my drives with usuall stuff like mp3s,dvds, ebooks etc.. (usuall pirate stuff :rolleyes::ph34r: )

[snip]

its been sponsored by my boss we are all ok to take it to the limits :D

[snip]

Sorry I just had to immortalize this.

Love how you openly state on the internet about your boss sponsoring you to use company funds to commit piracy. And take it "to the limits"

Edited by qasdfdsaq

Share this post


Link to post
Share on other sites

If budget is truly unlimited, and you want a robust, expandable system without any issues, why not buy it from a tier one server vendor? Everything will be tested and guaranteed to work together, you've got a warranty for the whole system, and there are plenty of options for expansion. The only disadvantage is price, and arguably flexibility. If you're an expert tweaker, and selector of components, you might be able to improve on what they offer, build it to be more flexible, or similar. But if price isn't an issue, why not let someone else do all the research and interoperability testing for you?

This is only a suggestion, I can appreciate if you want to overcome the technical challenge yourself just for the fun of it, or if the budget is unlimited but you'd feel you weren't getting good value out of a tier one.

Since I'm most familiar with HP, I'll give you an HP example. The HP StorageWorks 600 Modular Disk System takes 70 3.5" SAS or SATA hot plug drives in a 5U enclosure. $8500 with two dual port SAS controllers and redundant hot plug PSUs anf fans. $111 for four mini-SAS cables to hook up to a server. 2TB SAS drives are $949 (SATA only $50 less).

And then you'll need a server to present all that storage to the outside world. Since we're not skimping, how about a nice DL380 G7 performance model with 2 X5650s, 12 GB of RAM, 1 GB of flash backed write cache on the SAS controller, redundant hot plug fans and PSUs of course. $6789 plus half a dozen more 2GB dimms at $120 each. A pair of 60 GB SSDs (Samsung SLC based, but we're in this for reliability more than speed) - $1099 each. A couple more Smart Array P411 1G FBWC cards to connect to the 600 MDS - $799 each. An S10614 14U rack - $999. I could go on - ILO Advanced licenses, rack KVM units, PDUs, UPSes, but this will give you some idea. Oh, and you'll need an OS.

So, $11305 for the server, $8500 for the enclosure, $47450 for the drives, $999 for the rack makes $68254. That's RRP, you'll probably be able to get a significant discount ordering that much in one go. HP will assemble it all for you and ship the complete rack to you if you pay them to. I expect you'd get comparable options from IBM, Dell etc, though they may not have an exact equivalent for that storage unit.

Almost certainly far more expensive then a roll-your-own equivalent, but much better integrated, easier to build and service, and frankly more likely to work first time than a white box build. Not as much fun, perhaps, but a lot less hassle if you want something that just works. And on this scale, there's a lot more you can get wrong. Power, cooling, redundancy, compatability between the OS, controllers, backplanes and drives. Even the mechanics of servicing, getting cool air to the right place... and you can still get it wrong with the HP kit if you put it on a weak floor! Do bear in mind that this is going to weigh something like 200 - 250 kg including the rack and all the components. And that's not counting whatever you use for backups!

And you might need something more than a standard mains outlet to power this thing, and possibly air conditioning as well. Whatever you build, 50 drives are going to consume a lot of power and give off a lot of heat.

So, there you go. Is your boss willing to sponsor you to the tune of $70,000? :rolleyes:

Edited by Spod

Share this post


Link to post
Share on other sites

Much of that $70k quote wasn't too bad, except for the drives. There was a Newegg deal recently for 2TB green drives for $99 each with free shipping, which would cut the cost of drives to $4950. Granted, they would be much slower, but they'd also use much less power (which might make cheaper power supplies practical). Throw in another $5k for case, motherboard, controllers, and you're still only to $10k. Buy a second one for a backup, and get to $20k.

What makes this "robust and expandable" is that you can purchase more for redundancy, and replace the whole dang thing every year with a bigger model for less than buying a name brand model.

Share this post


Link to post
Share on other sites

Hi all,

I'm considering to build a new home server which will hold 50x2TB drives. As a case, I'm considering 9U/50 bay chembro. But I still dont know how I'll able to connect more than sata drives that mainboard already supports? For example, I can connect 4 sata drives to my mainboard but for connecting more than 4 drives, what should I do? sata port multiplier? I really dont know how its being done, if someone can point me a way I'd be really appreciate it. .

Have you considered another option, like "splitting" the whole storage into smaller (6 disk or so... cheap cards etc etc) units, and interconnect using iScsi?

Simple supermicro atom 510 dual lan cards would be just fine, not killer performance but enough for home server usage.

Bye!

Share this post


Link to post
Share on other sites

Have you considered another option, like "splitting" the whole storage into smaller (6 disk or so... cheap cards etc etc) units, and interconnect using iScsi?

Simple supermicro atom 510 dual lan cards would be just fine, not killer performance but enough for home server usage.

Bye!

Well just to be more precise, I don't mean your idea is wrong. But any complex, expensive setup eventually will run out of space (even for lack of expandability). You can further expand if you distribute your storage through multiple hosts (using iSCSI). No need to have multiple samba shares, if that is an issue: you can even join drives from different hosts with mdadm or lvm, or (more safely) with aufs on the main rig (say the master node).

But any of you hypothetical "slaves" can be anything you want, even a 9u/50 bay unit.

Or, if you feel adventurous, you can take a look at gfs, glusterfs or so...

Just my two cents :-)

Edited by GioF_71

Share this post


Link to post
Share on other sites

What GioF_71 said-- don't forget about future needs and expandability!

The biggest, most badass thing might not be the best idea in the long run.

And just for fun there's always Backblaze... keep in mind they run desktop drives in there so suitability may be a little iffy (at least in my book, I would not do it. But that's just me... others get it to work, but then yet others don't...).

http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now