Jump to content


Photo

Need help with RAID solution


  • You cannot start a new topic
  • Please log in to reply
21 replies to this topic

#1 dwb1729

dwb1729

    Member

  • Member
  • 10 posts

Posted 11 April 2014 - 06:03 AM

I need a 12 (at least) drive RAID solution.

 

Up till now I've been using Mediasonic's 8-bay RAID enclosure with 8 Seagate 3 TB's in a RAID 50 setup. This gives 18 TB of space (16.3 TiB) It's currently down to 10% available space so it's time to get something bigger.

 

I'd like to setup something like a 12-drive RAID 6 with 4 TB drives.

 

As much as I look I can't find a 12-drive RAID enclosure that isn't ten times the cost of Mediasonic's great 8-drive RAID enclosure.

 

I can't find any reliable RAID cards for building my own NAS.

 

Does anyone have any recommendations?

 

RAID 6 provides the fault tolerance I need.

4 TB drives provide the capacity I need.

It only needs to be able to stream bluray content and occasionally mount ISO's from it. I don't think any solution is going to fall short of being able to stream compressed bluray content to 2 or 3 peers simultaneously.


#2 Brian

Brian

    SR Admin

  • Admin
  • 5,158 posts

Posted 11 April 2014 - 01:46 PM

There are a lot of options but it sounds like budget is driving most of the considerations? Can you tell us more about the budget, it doesn't look like performance is going to be an issue once you get that many spindles involved. 


Brian

Publisher- StorageReview.com
Twitter - @StorageReview

 

#3 continuum

continuum

    Mod

  • Mod
  • 3,536 posts

Posted 11 April 2014 - 04:36 PM

FWIW, Seagate nearline 5TB and 6TB drives are available, if that helps anything. For the $329 for a Mediasonic 8-bay I don't think you're going to find anything cheaper, at least not enclosure-wise.

 

Synology and others do support expansion chassis for some of their models but as you probably already found, those aren't cheap. You'd have to do the math on whether or not more drive bays is worth the cost vs. fewer higher-capacity drives.

 

Hardware-wise getting 16 bays and building your own is doable but still probably 3x or 4x or 5x the cost of a $329 Mediasonic 8-bay...


#4 dwb1729

dwb1729

    Member

  • Member
  • 10 posts

Posted 11 April 2014 - 06:21 PM

Budget isn't really an issue. I have a few people going in on it with me.

 

So I'm just curious what the options are for a 12-bay RAID 6 enclosure of 4 TB drives.

 

I'm seeing stuff from NORCO, SANS DIGITAL, and others.

 

Trying to figure out if I should buy one of these $1k+ enclosures and an expensive RAID card and hook the whole thing up to my PC (or a new cheap PC) via a SAS cable.

-or-

If it would be possible to avoid a ton of that cost by just building a cheap computer in a huge tower that can house 12 (or 13, including boot) drives and just hook them all up to an internal RAID card. (is this possible?)

 

If that second option is recommended, how exactly would it work? What do all the hard drives plug into? Is there some extra device I'd need to buy to plug all the hard drives into that would then connect to the RAID card via a SAS cable?

 

You can see my head is swimming with all the options here. Limited knowledge of all this stuff doesn't help much either :(


#5 continuum

continuum

    Mod

  • Mod
  • 3,536 posts

Posted 11 April 2014 - 07:09 PM

Most cases top out around 8 harddisks, you can then buy 5.25" to 3.5" bay convertors to fit more 3.5" harddisks.

 

Most motherboards top out at 6 SATA ports, you can then buy PCI-e expansion cards with additional SATA ports-- either low-cost ones that basically just work as more SATA ports, then run software RAID of some sort, or you can fork out real money for a PCI-e hardware RAID expansion card. (Adaptec 8-series 16-port is about $1,100).

 

Cabling is a minor detail, don't worry about that. Most RAID cards today take some sort of SAS or mini-SAS cable, fanout cables to SATA ends are dirt cheap if that's what you need.

 

All depends how much performance you need, what tolerance for failure (aka reliability ;) ) you have, what your budget is, that sort of thing.


#6 dwb1729

dwb1729

    Member

  • Member
  • 10 posts

Posted 11 April 2014 - 09:25 PM

Well what I'm wondering is, say I found a tower that could house the cards, and I had a RAID controller plugged into the mobo inside. How would I hook the drives up to the RAID card? Do RAID cards have tons of ports? Or is there some other device I'd need to use into which I'd plug all the disks, that then connects to the RAID card via a SAS cable?

 

Performance requirement: should be able to pump 100 megs/second. That's plenty to have half a dozen (at most) people streaming compressed bluray content (that's in the neighborhood of 10 gigs tops per 7200'ish seconds which is only 1.5 MB/second) or mounting ISO's off of it or whatever. And reasonable for writes as well. And fast enough for rebuilds to not take forever in case of drive failure. I'm pretty confident that with 12 drives in any RAID config, I'm going to be able to get that bare minimum 100 megs/second, so I'm not worried about performance at all.

Fault tolerance requirement: I was considering doing a RAID 60 with two six-drive layers so it could absorb 4 disk failures as long as no more than two were from a given layer. But I believe RAID 6 will be sufficient, since it takes me from 2/3 space utilization to 5/6 and I'll have spare drives handy to deal with failures.

 

So what I know I want is: 12x4TB drives in a RAID 6 configuration.

What I'm still trying to decide is: what type of enclosure/NAS/computer-with-really-big-tower-that-can-house-many-drives/RAID-card/etc. do I want. So far enclosure seems to be the single most expensive component (all drives together being more expensive of course) and I figured maybe there's no real need for an enclosure as long as I've got a big tower that can house 12 drives all hooked up to a RAID card somehow. Just wondering how exactly they hook up or if I'm misunderstanding how RAID enclosures work. Having used the Mediasonic which handles everything, including RAID, means I've never had to actually construct a RAID array by hand before.


Edited by dwb1729, 11 April 2014 - 09:26 PM.

#7 dwb1729

dwb1729

    Member

  • Member
  • 10 posts

Posted 13 April 2014 - 06:23 PM

Does anyone know this? It's one of the major issues holding me back from pulling the trigger on these purchases


#8 continuum

continuum

    Mod

  • Mod
  • 3,536 posts

Posted 13 April 2014 - 11:51 PM

Do RAID cards have tons of ports? Or is there some other device I'd need to use into which I'd plug all the disks, that then connects to the RAID card via a SAS cable?

Go look at a retail box RAID card at a place like Newegg. You'll be able to find out exactly what's included for the specific card you're looking at. It will assauge your fears.

 

Or as I noted in my post above yours, appropriate cabling is readily available.

 

Performance requirement: should be able to pump 100 megs/second.
Running a parity RAID will require a little horsepower and enough spindles to keep things happy (since there's a write performance penalty with it), but you're talking 12 spindles, which is more than enough spindles that this won't be an issue.

 

and I figured maybe there's no real need for an enclosure as long as I've got a big tower that can house 12 drives
Finding regular cases that can hold more than 8x3.5" drives is tough, but you can always look at 5.25" to 3.5" bay adapters as well-- 3x5.25" to 4x3.5" adapters are pretty common, as are 3x5.25" to 5x3.5" adapters.

#9 dwb1729

dwb1729

    Member

  • Member
  • 10 posts

Posted 14 April 2014 - 07:05 AM

I scoured those links but I still don't understand what connects the HDD's to the RAID controller. It only shows two SAS ports and all the cables are 1:1. So I just don't understand:

 

What connects all the drives to the controller? I assumed the RAID controller would have 8 SATA ports. But it has two SAS ports.

 

I'm really confused about this :(

 

 

Is it that the drives need to be directly connected to the motherboard? If so what's the point of these SAS ports on the RAID controller anyway? I'm beginning to question whether a 12-drive RAID is possible without using an enclosure.

 

Or am I supposed to use some cable that fans out one SAS port to multiple SATA ports? I just want to do what's most standard here and not find some awkward workaround.


Edited by dwb1729, 14 April 2014 - 08:10 AM.

#10 anywhere

anywhere

    Member

  • Member
  • 63 posts

Posted 14 April 2014 - 07:57 AM

I made the near exact same setup and scenario of purpose. Samba/nfs media over network.

But I went ugly and hillbilly and spot welded 2 cases together so I have the extra width for hdds and airflow/fans. I have brakes and tig equipment, so I made a nice pretty aluminum setup from scraps around the shop. After that was a trip for powder coating and it's sexy.

I cut out the 3.5" bays and welded in 5.25" rails from old cases., so I can use those cooler master 3 5.25 to 4x3.5 Bay converters with a fan. All together of both cases and the 6, and the factory 3 3.25, I have 30 3tb in raid 6 spanned over 3 pci express hbas and the on board sata.

512mb on a 2.8 amd running mdadm on debian. Parity writes brings that poor cpu to its knees. It does make 300mb reads and 170mb writes. But I'm 100 percent sure if I had more raw cpu cycles to crunch #s it's be better. Next year might find something better in the dumpster. I'll keep patient.

Boot drive is a $6 4gb USB pen drive with swap disabled.
Dual nics , I got tired of saturating my 'media' network every time I started a 2 day 2tb dump/copy from my seed box onto it to the point nothing would respond. (too cheap to go gigabit switches, and these 10/100s won't die.) So I did a direct crossover cable to a 2nd nic when the seed box visits once a month.


Software raid has been nice. Add drives, grow array, grow filesystem. Done. Rinse and repeat. It Reshapes the arrau around 130mb/sec.

Today will be the first test of mdadm and the raid5 write hole. Lost power in the middle of copying a 2tb visit. No ups, but I have the array with a 512mb bitmap write-intent enabled. I'll keep posted.



Sent from my rooted HTC Supersonic using Tapatalk 2 Pro


#11 dwb1729

dwb1729

    Member

  • Member
  • 10 posts

Posted 14 April 2014 - 08:48 AM

How do I hook the HDD's to the RAID controller?


#12 anywhere

anywhere

    Member

  • Member
  • 63 posts

Posted 14 April 2014 - 09:46 AM

I used just sata cables? Hdd to hba. Zip ties to make things tight and opens air flow.

Then bought custom power molex cut to fit/diy hobby style. One lead has 4 crimped in a line so it's neat and fits the enclosures hdd space perfectly.


Sent from my rooted HTC Supersonic using Tapatalk 2 Pro


#13 dwb1729

dwb1729

    Member

  • Member
  • 10 posts

Posted 14 April 2014 - 09:48 AM

I'm talking about a RAID controller though. Not an HBA card.

 

HBA just effectively expands the # of SATA slots on your mobo right?

 

Beginning to think there is some conspiracy on the internet to avoid answering this question. I've asked in so many forums and fired off so many emails and read so much and I just cannot find an answer to the question of how you plug HDD's into the RAID controller. I know I'm overlooking something but at this point, considering how many times I've asked that specific question in this thread and no one seems willing to answer.. I'm going to guess maybe.. no one knows? -.-


Edited by dwb1729, 14 April 2014 - 11:56 AM.

#14 anywhere

anywhere

    Member

  • Member
  • 63 posts

Posted 14 April 2014 - 01:00 PM

Depends if you're fancy and using a backplane.

One of my card are actually raid, but disabled and jbod mode, has 2 SAS ports for 8 total and power, and the data and power are one connector. Then there's a pigtail where it draws 5 and 12v.

The other cards are $20 special controllers. I went cheap. Works like the expensive stuff. more flexible too.

Sent from my rooted HTC Supersonic using Tapatalk 2 Pro


#15 continuum

continuum

    Mod

  • Mod
  • 3,536 posts

Posted 14 April 2014 - 02:21 PM

I scoured those links but I still don't understand what connects the HDD's to the RAID controller. It only shows two SAS ports and all the cables are 1:1. So I just don't understand:
You didn't look very hard at the links above.

 

e.g.:


Adaptec ACK-I-HDmSAS-4SATA-SB-.8M cable-2279800-R.jpg

The Adaptec ACK-I-HDmSAS-4SATA-SB-.8M is an internal Mini Serial Attached SCSI HD x4 (SFF-8643) to (4) x1 Serial ATA (adapter based) fan-out cable with sideband signals (SFF-8448). It measures 0.8 meters and is used for connecting a SAS/SATA adapter to SATA disks, or a SAS/SATA backplane.

  • Part Number: 2279800-R
  • Suggested Retail Price: $19.00
  • Dimension: 0.8 Meters

#16 dwb1729

dwb1729

    Member

  • Member
  • 10 posts

Posted 14 April 2014 - 03:54 PM

I see. Thanks for the clarification. Not sure why I didn't see that. I went down the list looking for something like that (some kind of fan-out cable) and thought I only saw 1:1's. My mistake.

 

Would you recommend any particular brand for RAID controllers? I'm looking for a 12-port but of course if there are more, that is ok since I don't need to use them all.

 

I've seen a lot from Norco and San Digital that looked good (price per port)-wise. But I have no experience with them


#17 Brian

Brian

    SR Admin

  • Admin
  • 5,158 posts

Posted 14 April 2014 - 04:12 PM

We don't do much with the lower-cost alternatives on RAID card testing, but we're expanding. Just got some new Areca cards in we're evaluating to go with the standard standbys of LSI and Adaptec. 


Brian

Publisher- StorageReview.com
Twitter - @StorageReview

 

#18 dwb1729

dwb1729

    Member

  • Member
  • 10 posts

Posted 14 April 2014 - 04:44 PM

I'd be curious what your recommendations would be among the cards you have tested. I don't necessarily need to go with low cost, unless all else is equal


#19 Brian

Brian

    SR Admin

  • Admin
  • 5,158 posts

Posted 14 April 2014 - 06:26 PM

It would be LSI for this use case, Adaptec's newer stuff is really good but not relevant here. 


Brian

Publisher- StorageReview.com
Twitter - @StorageReview

 

#20 dwb1729

dwb1729

    Member

  • Member
  • 10 posts

Posted 14 April 2014 - 10:03 PM

Cool I'll check them out

 

And I see why you didn't bother mentioning them :P

 

I wouldn't even be able to get a quote on price without representing a company. Oh well. Their loss.


Edited by dwb1729, 14 April 2014 - 10:04 PM.

#21 continuum

continuum

    Mod

  • Mod
  • 3,536 posts

Posted 16 April 2014 - 04:17 PM

If you need real hardware RAID then as noted, you have to pay to play-- a 16-port hardware RAID such as an Adaptec 81605ZQ, Areca ARC-1883ix-16, or equivalent from LSI is going to be in the price range of $1k just for the card.

 

You can pick 'em up from CDW, Newegg, Provantage, Buy.com, Amazon, etc. No need to contact the company directly for a quote, no point in doing such a crazy thing as an end-user. ;)

 

Honestly if you're budget sensitive look into NAS4Free or FreeNAS as an OS that can do Linux software RAID for you.

 

I've seen a lot from Norco and San Digital that looked good (price per port)-wise. But I have no experience with them

Norco doesn't make their own RAID cards, at least not of the same calibre as Areca, Adaptec, or LSI. They don't even rebrand good stuff-- they rebrand low-end crap from Silicon Image or others. It might work fine, but honestly, if you have the skill, running FreeNAS or Nas4Free is a better alternative than low-end "RAID" products.


#22 anywhere

anywhere

    Member

  • Member
  • 63 posts

Posted 17 April 2014 - 01:54 PM

If you need real hardware RAID then as noted, you have to pay to play-- a 16-port hardware RAID such as an Adaptec 81605ZQ, Areca ARC-1883ix-16, or equivalent from LSI is going to be in the price range of $1k just for the card.
 
You can pick 'em up from CDW, Newegg, Provantage, Buy.com, Amazon, etc. No need to contact the company directly for a quote, no point in doing such a crazy thing as an end-user. ;)
 
Honestly if you're budget sensitive look into NAS4Free or FreeNAS as an OS that can do Linux software RAID for you.
 
Norco doesn't make their own RAID cards, at least not of the same calibre as Areca, Adaptec, or LSI. They don't even rebrand good stuff-- they rebrand low-end crap from Silicon Image or others. It might work fine, but honestly, if you have the skill, running FreeNAS or Nas4Free is a better alternative than low-end "RAID" products.



I did 32bit with mdadm because of free hand-me-downs.

If you got nice 64bit hardware, zfs on freebsd, raidz2 or raidz3. Self healing from the parity write hole eliminates expensive controllers.

My Mdadm survived a nasty power outage on 20th in raid6. While copying to array from 4 different sources combined around 250mb/sec.

I used a 512mb write intent bitmap. No issues.

My experiences.


Sent from my rooted HTC Supersonic using Tapatalk 2 Pro





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users