Jump to content


Photo

Supermicro array questions


  • You cannot start a new topic
  • Please log in to reply
11 replies to this topic

#1 Brian

Brian

    SR Admin

  • Admin
  • 5,135 posts

Posted 01 December 2013 - 04:21 PM

Via email from Seth:

 

I'm looking at setting up a new storage array for myself, and find myself baffled with the choices of supermicro chassis to choose from, with the A models of fully connected drives to the E16 and E26 models with one and two expander backplanes. I want to know some simple things that are too simple to be documented anywhere: can sata drives do multipath? Would multipath on the E26 allow higher speeds and redundancy than the E16? Or only redundancy?

Note that I am phrasing the questions in terms of  specific hardware only because I lack the vocabulary and knowledge to describe my questions well. A good guide that is hardware agnostic would be ideal.

I'm in over my head, but I really do want the ease of building a six or nine drive array now and growing to nine or twelve drives in the future... And the gear doesn't seem too silly expensive when adding another dozen drives..

Thanks!

 

 


Brian

Publisher- StorageReview.com
Twitter - @StorageReview

 

#2 Kevin OBrien

Kevin OBrien

    StorageReview Editor

  • Admin
  • 1,406 posts

Posted 01 December 2013 - 08:47 PM

I'm not completely up with Supermicro model names, so I'll need a bit of help to know which models you're looking at specifically.

 

On the SATA question specifically, it won't do multipathing unless you add in an interposer card. SAS is the only standard that will do that natively. In regards to performance, you will see very little or no performance difference with multipathing. The main use for it is redundancy in the event of a hardware failure.


#3 compwizrd

compwizrd

    Member

  • Member
  • 411 posts

Posted 02 December 2013 - 01:53 AM

I have one of the 846 cases at work with the E16 in it.. 24 bays, single 8087 uplink.  This E16/E26 is a SAS expander, not a SATA expander.  Having said that, you can attach SATA drives to it if your controller supports them.. works fine with 24 SATA drives attached to a 4 port 3ware 9750-4i via the 8087 cable giving a 24 gbit link. (4 x 6gbit), which is enough for the spinning rust drives.


#4 sarnold

sarnold

    Member

  • Member
  • 2 posts

Posted 04 December 2013 - 09:41 PM

Thanks Brian, Kevin, and compwizrd,

 

Within the SuperMicro lineup, e.g. the SC846 or SC847 line, there are different models; the A models have a SAS port on the backplanes for every drive, the E16 models have a single 8087 connector for four SAS ports, the E26 models have two 8087 connectors on multipath-capable backplanes for eight SAS ports. (I think up to this point my understanding is solid enough.)

 

I'd really like to use SATA drives because they fit my budget better.

 

I don't think I need the performance that I could get from the A models -- I certainly don't need 24 in-flight reads or writes at once. I wonder if there is value to me in buying the E26 model, because there's a lot I just don't understand:

 

- Can SATA drives do multipath on backplanes like this?

 

- Can more than four SATA drives on one of these backplanes see performance improvements with multipath? i.e. could I have eight reads/writes in-flight on one of these E26 backplanes?

 

I don't know where I'd go to find the information I want, my usual sources (wikipedia and manpages) just don't cover things this far above normal consumer grade equipment, and most of the more advanced sources assume this is all introductory material. :)

 

Thanks!


#5 compwizrd

compwizrd

    Member

  • Member
  • 411 posts

Posted 04 December 2013 - 11:33 PM

You can use the SATA drives, but you need a SAS controller that will talk to expanders.  These are SAS expanders, not SATA multipath units.   There's a SAS multipath, but it's not the same thing

 

I've had 20 SATA drives live at the same time on the E16, and it maxes out around 2400 mbit

 

 

The E26 would let you add another drive chassis full of drives, with a single uplink from that chassis going to the expander.. though it might be easier to just have an 8 port SAS 4i4e card.

 

My home server has an Intel 20 port expander attached to a 4 port 3ware 9750.. was a lot cheaper than the supermicro method.


#6 sarnold

sarnold

    Member

  • Member
  • 2 posts

Posted 06 December 2013 - 09:06 PM

compwizrd, thanks for the response, I realize now that I've not noticed the distinction between the EL1 and EL2 backplanes vs the E16 and E26 chassis types.

 

I certainly don't need a second chassis. :)

 

While trying to reconcile this all, I did spot a paragraph in the SC847J.pdf that I don't recall reading before, which says in part that "no SATA drives should be used in dual expander backplanes". This alone reduces the problem space drastically. (I had seen a mention on Wikipedia that 6Gb/s SATA drives _could_ do multipath, but I haven't found any documentation anywhere else to corroborate this.)

 

Thanks for sharing some information about your personal setup. Do you have it documented more fully somewhere else? If not, may I ask for some more details? :) What case, power supplies, motherboard, and SAS expander are you using? Would you pick them again?

 

(I've started looking with supermicro because easy hot-swap bays and redundant power supplies seems like wonderful things even if I don't need to use them often. But perhaps some conveniences could be compromised for price.)

 

Thanks for the help.


#7 compwizrd

compwizrd

    Member

  • Member
  • 411 posts

Posted 06 December 2013 - 09:42 PM

Antec Titan 650 case, antec 430W power supply (which is probably overkill), Intel RES2SV240 sas expander(which seems to have gone way up in price, i think i paid about 200) .   supermicro cse-m35t for a 5 3.5" drive in 3 5.25" bay adapter.    10 WD Green 2 TB's in Raid6, off the 3ware 9750-4i.    2TB WD black boot drive.    Case is full.

 

Motherboard is an ASUS P8H77-V, Pentium G630 cpu, 8 gig ram.  I have an I3-2100 in a media server out front so if I need more power I can just swap them.  The server draws around 100W at idle.

 

Has no difficulty keeping up with anything I need, network copies run at about 115 mbyte/sec to/from the Windows machines.  Reads locally are in the 400 mark.

 

Things I'd do different:

 

I'd go with a quiet rack case with more bays.  Maybe even one of the cheap 3U/4U Norco units.  Supermicro is much nicer though.

WD Red instead of WD Green.  Red's didn't exist when I put this together.  Green is a really bad idea because of WD crippling the firmware on them.

 

Probably start with 4TB drives instead of 2's, I've upgraded this thing twice already.


#8 dietrc70

dietrc70

    Member

  • Member
  • 106 posts

Posted 09 December 2013 - 04:18 AM

Yikes--WD Greens in RAID 6 sound like a disaster waiting to happen!  I'm impressed that they have been working so well.  I'm also surprised that they play nice with your SAS expander.  I wouldn't have expected that.  You learn something new every day.

 

It's just as well, I think, that you didn't get 4GB Greens :)  I've noticed a great increase in angry reviews about the 3 and 4TB greens as opposed to the 2TB models.

 

Be sure to check out the new WD Se series when you need new drives.  I haven't used them (I use RE's) but they are priced pretty well and fully RAID qualified.  Supposedly the Reds are not so well suited for large arrays--weaker vibration compensation and tolerance for heavy use.  You'd probably get amazing performance with SAS drives!


Edited by dietrc70, 09 December 2013 - 04:20 AM.

#9 compwizrd

compwizrd

    Member

  • Member
  • 411 posts

Posted 09 December 2013 - 10:17 AM

Luckily my greens were old enough I could apply the head parking fix.  I've had a few bad sectors but the drives themselves have stayed stable with the usual expected death rate.

 

I have a WD Re sitting as a hot spare on one of my machines at work.  Sits there blinking the chassis red light at me, predicting a failure according to the manual.  Only blinks the red light at me when it's setup as a hot spare.  I should find another drive and put it in a raid1 and see if it stops blinking.  Smart tests fine, and so does beating it up repeatedly.

 

I've used Hitachi drives in both 5400 and 7200 rpm drives in 10+ tb raid6 arrays and had no problems, it's too bad HGST has so many supply problems, I've liked them.

 

SAS would completely shatter the budget. :)


#10 dietrc70

dietrc70

    Member

  • Member
  • 106 posts

Posted 09 December 2013 - 02:38 PM

The blinking is probably normal, I'm just surprised it isn't a green led blinking.  I have a managed enclosure, and I recall my Adaptec making hot spares blink green.  Maybe it's some quirk in your enclosure setup that causes the red light to blink instead.  Is the Re an SAS drive?  If the other drives are SATA the controller might be complaining about that.


#11 compwizrd

compwizrd

    Member

  • Member
  • 411 posts

Posted 09 December 2013 - 03:08 PM

Blue for activity, solid red for "you should have replaced me a week ago" and apparently blinking red is "either panic, or this is your hot spare"

 

The RE is SATA.  It's even on the 3ware HCL.

 

If I can find it, I'll put my other spare drive in and see what it does.


#12 compwizrd

compwizrd

    Member

  • Member
  • 411 posts

Posted 12 December 2013 - 03:46 PM

Found my other drive, and so either WD quality control has gone downhill, or my enclosure blinks the red light twice for Hot Spare.





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users