[poodel]

My old file server is reaching it's limits, so I'm building a new one. This time I'm going to make it proper. So far I've got a 19" Chieftec case big enough to fit 18 HDD:s, some hotswap cassettes, and now I'm looking at the storage. Having worked with storage in Sun environment for ~10 years, I'm pretty sure of what I want, but not as sure of how the consumer-level hardware will perform.

What I'm aiming for is:
* RAID5 redundancy
* Single file system on the array. The 2TB limit should not apply if I run a 64-bit debian distro?
* Online capacity expansion... add a disk whenever I need to, and have the file system grow.
* Debian compatibility. Which filesystem do I use to make it growable on the fly?
* RAID1 snapshot of the root disk (in case some APT update screws something important up)

I'm not going to boot from the RAID array, so I'll need some smaller SATA disks as root disks.

Disks
Bigger is better, so I'm thinking 750GB Seagates. Any reason not to?

RAID controller
I've narrowed it down to Promise EX8350 or Areca 1220, but here's where it gets tricky. Does anyone know how they work in Debian? I'm a bit worried about what's possible to do and what's convenient to do.

[jpiszcz]

poodel, on Jan 1 2007, 07:10 AM, said:

My old file server is reaching it's limits, so I'm building a new one. This time I'm going to make it proper. So far I've got a 19" Chieftec case big enough to fit 18 HDD:s, some hotswap cassettes, and now I'm looking at the storage. Having worked with storage in Sun environment for ~10 years, I'm pretty sure of what I want, but not as sure of how the consumer-level hardware will perform.

What I'm aiming for is:
* RAID5 redundancy
* Single file system on the array. The 2TB limit should not apply if I run a 64-bit debian distro?
* Online capacity expansion... add a disk whenever I need to, and have the file system grow.
* Debian compatibility. Which filesystem do I use to make it growable on the fly?
* RAID1 snapshot of the root disk (in case some APT update screws something important up)

I'm not going to boot from the RAID array, so I'll need some smaller SATA disks as root disks.

Disks
Bigger is better, so I'm thinking 750GB Seagates. Any reason not to?

RAID controller
I've narrowed it down to Promise EX8350 or Areca 1220, but here's where it gets tricky. Does anyone know how they work in Debian? I'm a bit worried about what's possible to do and what's convenient to do.

Get the 750GB ES (for RAID drives) and Areca no question, promise cards are 'fakeraid' - Areca cards are the fastest SATA cards in the world.

[jpp]

Have you looked at the Infrant RAID NAS's? Here's a link to their Wiki site. That is of course, if your interest is solely Raid 0-5 storage inc hot swapping.

Phil.

[qasdfdsaq]

If cost is a concern over pure capacity, I'd suggest getting something other than 750GB drives. They've the worst capacity/price ratio of any 7200RPM drive out there.

[jpp]

qasdfdsaq, on Jan 1 2007, 11:45 PM, said:

If cost is a concern over pure capacity, I'd suggest getting something other than 750GB drives. They've the worst capacity/price ratio of any 7200RPM drive out there.

Also remember that if you choose RAID5 with X-Raid (Infrant one company that has this feature - others no doubt as well) you can upgrade one disk at the time without having to backup and restore. Sweet spot right now is the Seagate 320GB SATA II ST3320620AS at about US$75 eac. But, realise that until you have all 4 disks upgraded (to say 750GB when these become decent value per GB), you will be limited by the size of your smallest disk - that is if you want to retain RAID 5. As you know, the useful data storage is the total capaity of all drives less one, i,e 960GB for 4 x 320GB drives).

[poodel]

jpiszcz, on Jan 2 2007, 01:26 AM, said:

Get the 750GB ES (for RAID drives) and Areca no question, promise cards are 'fakeraid' - Areca cards are the fastest SATA cards in the world.

The ES drives are about $80 more a piece, so what's the upside?

[jpiszcz]

poodel, on Jan 2 2007, 05:24 AM, said:

jpiszcz, on Jan 2 2007, 01:26 AM, said:

Get the 750GB ES (for RAID drives) and Areca no question, promise cards are 'fakeraid' - Areca cards are the fastest SATA cards in the world.

The ES drives are about $80 more a piece, so what's the upside?

Depends on how reliable you want your system to be-- the consumer-- regular drives (which I currently use 7200.8/9/10 400s) will often die/have more problems if they encounter a few bad sectors. Where with RAID (ES) or WD RE/RE2 drives, they will just keep ticking/running with thousands of bad sectors (over time) because they are built to run in a RAID setting.

[poodel]

qasdfdsaq, on Jan 2 2007, 05:45 AM, said:

If cost is a concern over pure capacity, I'd suggest getting something other than 750GB drives. They've the worst capacity/price ratio of any 7200RPM drive out there.

Yes, but then there's the RAID controller, the disk enclosures, the computer, the power usage... assuming I start with 4 * 750, the initial cost of the disks is going to be about 40% of the total server cost. Considering the whole system, !/$ would actually be worse if I went for the cheaper disks.

This post has been edited by poodel: 02 January 2007 - 09:41 AM

[jboles]

Also don't forget the 32-bit LBA (2TiB) limit is not a function of the OS but of the RAID controller. Buying one new, however, it shouldn't be an issue at all.

[Haversian]

jboles, on Jan 2 2007, 02:31 PM, said:

Also don't forget the 32-bit LBA (2TiB) limit is not a function of the OS but of the RAID controller. Buying one new, however, it shouldn't be an issue at all.

True.

But the LBA limitation is not the only (nor even the most important) limitation. The filesystem may have limitations too, though anything reasonably modern won't. I'm a fan of XFS (wikipedia), but even ext2 (with 8k blocks) will scale to 32TB filesystems and 2TB files.

The biggest limitation in terms of using not-cutting-edge tech will be the partition table format. The standard partition formats limit you to 2TB as well, so you'll have to use something like GPT (wikipedia) or simply put a filesystem on unpartitioned space, which works just fine.