[cdh]

I hope so much that someone here can help me (very worried). I was having trouble with my RAID5 recently. Drives were coming up as failed or missing. I was able to get them all back except for two...one said failed, and the other said missing. However, I tried many times to remove and re-insert the missing drive. It would not come back up. So, after reading online, it seemed an option was to delete the RAID set and then re-create it. I did so, and my D and E drives were both visible as they should have been. The D drive seemed to work also. However, the E drive did not work, which is the vast majority of the content (the D drive is about 750 GB, and then the E drive is the rest of the 15TB). I remembered that I didn't select the "Greater than 2TB option when I re-created that RAID set (I thought the option was only if your operating system did not support more than 2TB...mine does...Windows XP x64). Since the other one worked, though, and it was not greater than 2TB, I figured that's what it must be. So, I went into the volume set options and changed that setting. It didn't say it was going to erase the disk contents, unless I missed it (I have 3% of my vision...so, it is possible, although I was looking very closely since this is extremely important to me). Yet, it began initializing, which to me means bad news. It was at about 15% by the time I got into the web admin to see what was happening now. I shut down the computer, turned off the drive boxes, and then rebooted so that I could send this message. Firstly, is it indeed erasing all my data? If so, can it be aborted/reverted at this point (long shot, I guess, but I am hopeful!)? I very much hope someone here will have good news for me. Thanks much in advance.

[continuum]

There are software tools that exist to recover damaged RAIDsets, but a 16-disk RAID5 that;'s already been 15% initialized?? Eeeeeek. You have probably just screwed yourself...

(and as you experienced, rebuild times on arrays this large get really scary, you have a HUGE window of vulnerability to a 2nd disk failure, which hoses you pretty good. I would, in the future, run it as a pair of 8-disk RAID6's at least..)

[HachavBanav]

RAID 5 on large SATA drives is NOT reliable. Full point.
RAID 6 on more than 10 large SATA drives is NOT reliable as well.

Use RAID 10 and Enterprise class SATA drives (with a 1 per 10E15 BER) to avoid those kind of problems !

[HachavBanav]

Just to understand how BAD is a 1TB SATA with a 1 per 10E14 BER in a raid array.

A BER/UBE of 1 per 10E14 means a 8.8% probability of ONE unreadable sector per 1TB read !
==> You have 16 of those 1TB unreliable HDD !

Enterprise class SATA drives (and large SAS HDD) are one order of magnitude more reliable : 1 per 10E15
Most small SAS HDD are 1 per 10E16

[qasdfdsaq]

HachavBanav, on Nov 9 2009, 10:22 AM, said:

Just to understand how BAD is a 1TB SATA with a 1 per 10E14 BER in a raid array.

A BER/UBE of 1 per 10E14 means a 8.8% probability of ONE unreadable sector per 1TB read !
==> You have 16 of those 1TB unreliable HDD !

Enterprise class SATA drives (and large SAS HDD) are one order of magnitude more reliable : 1 per 10E15
Most small SAS HDD are 1 per 10E16

Many non-enterprise class SATA drives have a rating of 1 sector in 10^15.

Also, rated values != reality.

Rated values == marketing.


1.5TB x5 = 7.5TB

Zero unreadable sectors in over 100TB read.

Explain that.

[HachavBanav]

qasdfdsaq, on Nov 16 2009, 02:17 AM, said:

Many non-enterprise class SATA drives have a rating of 1 sector in 10^15.

You are welcome to keep buying Desktop class drives.

qasdfdsaq, on Nov 16 2009, 02:17 AM, said:

Zero unreadable sectors in over 100TB read.

To me, the UBE/BER point is more sensitive at REBUILD time where the HBA reads the ALL drive at once.

[qasdfdsaq]

zpool scrub is a read-verify of the entire drive at once. I do it daily.

You've still offered no explanation.

This post has been edited by qasdfdsaq: 18 November 2009 - 08:22 PM

[HachavBanav]

qasdfdsaq, on Nov 16 2009, 02:17 AM, said:

Zero unreadable sectors in over 100TB read.

Can you check your 5x 1.5TB drives SMART data and let us know ?

[TRACKER_MAN]

qasdfdsaq, on Nov 18 2009, 08:21 PM, said:

zpool scrub is a read-verify of the entire drive at once. I do it daily.

You've still offered no explanation.

I can confirm observations on qasdfdsaq with opensolaris / zpool scrub. No problems at all.

[HachavBanav]

TRACKER_MAN, on Nov 19 2009, 03:25 PM, said:

I can confirm observations on qasdfdsaq with opensolaris / zpool scrub. No problems at all.

Well, can you tell us how many sectors did the scrubber fixed ?