Sign in to follow this  
lleo530

48 tb server help

Recommended Posts

hello, im planning on building a 48 tb server for movie storage.

this would most likely be the configuration

1 areca ARC-1680IX-24-4G PCI-Express x8 SAS RAID Card

24 Seagate Constellation ES 2TB 3.5" SATA 3.0Gb/s Enterprise Internal Hard Drive -Bare Drive

1 SUPERMICRO MBD-X8DTi-F-O Extended ATX Server Motherboard

2 Intel Xeon E5506 2.13GHz LGA 1366 80W Quad-Core Server Processor

2 Crucial 6GB (3 x 2GB) 240-Pin DDR3 SDRAM Triple Channel Kit Server Memory Model CT3KIT25672BA1339

1 SUPERMICRO CSE-846TQ-R900B Black 4U Rackmount Server Case

this would be configured in raid 6 and windows server 2008

what i would like to know is what are the chances of a raid like this failing and what kind of environment should this equipment be placed in ?

Share this post


Link to post
Share on other sites

First off, sounds like an excellent rig, good luck with it - and welcome to the forums.

What do you mean exactly by "the chances of the RAID failing?"

And by environment, do you mean physical environment like a media closet or some such?

Share this post


Link to post
Share on other sites

You'll never be able to rebuild this if a drive dies because of URE. You'll need to make your raid size smaller.

so what raid do you suggest i should do to achieve 48tb in total ?

Share this post


Link to post
Share on other sites

First off, sounds like an excellent rig, good luck with it - and welcome to the forums.

What do you mean exactly by "the chances of the RAID failing?"

And by environment, do you mean physical environment like a media closet or some such?

like the probability of this raid failing and loosing all the data.

and environment i mean just that like a closet temperature humidity etc.

Share this post


Link to post
Share on other sites

I don't think there's a good way to handicap the RAID's chances of failure. I'd say they're very low, but other than that...you could reduce your risk by using two RAID cards. You could reduce your risk further by making two machines instead of one.

As to the closet humidity, do you have a separate AC unit for this room? A thermostat with humidistat solves your problem, but it's hard to conceive of a separate AC unit just for this. Can you detail the configuration a little more?

Share this post


Link to post
Share on other sites

so what raid do you suggest i should do to achieve 48tb in total ?

Instead of one big RAID make more smaller ones. If you're using RAID 5, make sure you're raid set is smaller than 12tb.

Share this post


Link to post
Share on other sites
like the probability of this raid failing and loosing all the data.
http://www.storagereview.com/guide/relRel.html

I'd say it's a real concern. :)

Generally I don't like to run arrays larger than 8 disks, and even then only in RAID6 as the probability of a failure during rebuild gets real scary, really fast with 2TB disks.

The implications of this are clear. If you create a RAID array with four drives, each of which has an MTBF figure of 500,000 hours, the MTBF of the array is only 125,000 hours! In fact, it's usually worse than that, because if you are using hardware RAID, you must also include the MTBF of the controller, which without the RAID functionality, wouldn't be needed. For sake of illustration, let's say the MTBF of the controller card is 300,000 hours.

If you want 48TB usable you're talking at least 32 disks in the array with RAID6... four arrays is what I would be using to minimize impacts of rebuild window vulnerability on ALL of your data.

Share this post


Link to post
Share on other sites

hello, im planning on building a 48 tb server for movie storage.

this would most likely be the configuration

1 areca ARC-1680IX-24-4G PCI-Express x8 SAS RAID Card

24 Seagate Constellation ES 2TB 3.5" SATA 3.0Gb/s Enterprise Internal Hard Drive -Bare Drive

1 SUPERMICRO MBD-X8DTi-F-O Extended ATX Server Motherboard

2 Intel Xeon E5506 2.13GHz LGA 1366 80W Quad-Core Server Processor

2 Crucial 6GB (3 x 2GB) 240-Pin DDR3 SDRAM Triple Channel Kit Server Memory Model CT3KIT25672BA1339

1 SUPERMICRO CSE-846TQ-R900B Black 4U Rackmount Server Case

this would be configured in raid 6 and windows server 2008

what i would like to know is what are the chances of a raid like this failing and what kind of environment should this equipment be placed in ?

Stop thinking of raid with that data size..

use WHS and duplicate the shares you need to be secured on more than one disk (data you cant afford to loose by disk failure).

if Data integrity is more important than that buy two boxes ;-)

Share this post


Link to post
Share on other sites

The thing is, you propably want all those nice drives to be seen as one drive on your server. So you dont have to split data. Dont know about WHS, but i use linux with software raid and LVM on top. That does the trix. Multi raid sets, one big LVM ;) you can read more about my setup in this thread (im about 6 reply's down)

But as the other guys mentioned. The rebuild times on 6drive 2tb+ is awfull, even with HW raid. I did loose my 6x1tb raid 5 once (had backup, so no biggy) because of disk faliure while rebuilding. So with 2tb drives, you have "about" double the rebuild time, but hey, you can accept one more disk faliure ;)

To sum up. Less drives in each raid6, combine logical on top. And remember your backup.... what if the house cought on fire ?

//Jan Chu

Share this post


Link to post
Share on other sites

How about a raid60? That would give you a stripe of two raid6 sets with 4 parity drives total, meaning you could lose up to two drives in each set before losing the array. The Areca arc-1680ix supports raid60. If you set aside 2 for hot spares you'd have around 36TB to work with.

Share this post


Link to post
Share on other sites

Raid60 is nice enough. But what do you do when all that storage is used ? How do you exspand ? what are the plans here.... With mdadm-lvm you can make your pool of raid6's and be flexible when you want to exspand. I know am aware that he can replace every drive in his raid60 1 at a time per raid6 (or 2 if he has GIANT balls :D ) and when all drives are replaced he can exspand his raid60. But that is not that flexible. Offcause he wouldnt get the performance with a pool of raid6 in lvm because of the lack of stripe.

Question: What kind of performance do you need ? What type of workload do you exspect ? (small files / big files , seq / random , read/write ) Is performance something that should be though of ?

I to was a firm beliver in HW raid, but i must say, i love my SW raid. and it performs very well (granted no BBU, so no write back) But im not in it for the performance, only storage and flexibility.

//Jan Chu

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this