[Haversian]

jpiszcz, on Jan 5 2007, 06:25 PM, said:

jpiszcz - Your mediocre write speeds are on par with what I've seen on other software raid5 solutions. I'm not so sure you're PCI limited, it's just that raid5 read-calculate-write thing that will make all writes slow.

If all your software RAID5 devices are behind the PCI bus, writing is going to be slow. I got about 35MB/sec on a 4-drive array with all drives on a PCI controller. If I moved half the drives to chipset-attached SATA ports, I could roughly double that. And it is the PCI bus that's the limiting factor. Assume 100MB/sec real-world PCI bandwidth, and zero computation time. With a 4-drive array, you're reading 3 stripes, computing, and writing the 4th stripe, for about 25MB/sec throughput. I presume the stripe cache deserves credit for write performance in excess of that figure.

My benchmarks (and other linux software raid5 info)

[jpiszcz]

Haversian, on Jan 6 2007, 01:41 AM, said:

jpiszcz, on Jan 5 2007, 06:25 PM, said:

jpiszcz - Your mediocre write speeds are on par with what I've seen on other software raid5 solutions. I'm not so sure you're PCI limited, it's just that raid5 read-calculate-write thing that will make all writes slow.

If all your software RAID5 devices are behind the PCI bus, writing is going to be slow. I got about 35MB/sec on a 4-drive array with all drives on a PCI controller. If I moved half the drives to chipset-attached SATA ports, I could roughly double that. And it is the PCI bus that's the limiting factor. Assume 100MB/sec real-world PCI bandwidth, and zero computation time. With a 4-drive array, you're reading 3 stripes, computing, and writing the 4th stripe, for about 25MB/sec throughput. I presume the stripe cache deserves credit for write performance in excess of that figure.

My benchmarks (and other linux software raid5 info)

Yes yes-- I know that, that is why I bought a new motherboard, disk controllers and the like, everything of which, including the gigabit ethernet chip will be on PCI-e, this should let me scale linearly, hopefully I will easily get in excess of 100-200MB/s.

Justin.

jpiszcz, on Jan 6 2007, 04:27 AM, said:

Haversian, on Jan 6 2007, 01:41 AM, said:

jpiszcz, on Jan 5 2007, 06:25 PM, said:

jpiszcz - Your mediocre write speeds are on par with what I've seen on other software raid5 solutions. I'm not so sure you're PCI limited, it's just that raid5 read-calculate-write thing that will make all writes slow.

If all your software RAID5 devices are behind the PCI bus, writing is going to be slow. I got about 35MB/sec on a 4-drive array with all drives on a PCI controller. If I moved half the drives to chipset-attached SATA ports, I could roughly double that. And it is the PCI bus that's the limiting factor. Assume 100MB/sec real-world PCI bandwidth, and zero computation time. With a 4-drive array, you're reading 3 stripes, computing, and writing the 4th stripe, for about 25MB/sec throughput. I presume the stripe cache deserves credit for write performance in excess of that figure.

My benchmarks (and other linux software raid5 info)

Yes yes-- I know that, that is why I bought a new motherboard, disk controllers and the like, everything of which, including the gigabit ethernet chip will be on PCI-e, this should let me scale linearly, hopefully I will easily get in excess of 100-200MB/s.

Justin.

Also, http://www.monkeysus...e%20server.html, I have read this before, nice writeup, whoever did it

[Haversian]

jpiszcz, on Jan 6 2007, 03:29 AM, said:

Yes yes-- I know that, that is why I bought a new motherboard, disk controllers and the like, everything of which, including the gigabit ethernet chip will be on PCI-e, this should let me scale linearly, hopefully I will easily get in excess of 100-200MB/s.

Probably. If real-world writes of roughly bus speed / 3 hold for PCIe as well as PCI, you should see about 70MB/sec writes for 4 drives on an x1 controller (or possibly somewhat more since PCI is bidirectional and PCIe has 250MB/sec each way per lane). I don't have much practical experience with how well various chipsets scale when you're using several PCIe lanes simultaneously, but if you use an x4 card, or 2 x1 cards, or an x1 card and chipset-attached SATA ports, I'd be surprised if you can't beat at least 100MB/sec sequential writes.

Good luck!

jpiszcz, on Jan 6 2007, 03:29 AM, said:

Also, http://www.monkeysus...e%20server.html, I have read this before, nice writeup, whoever did it

Thanks! I hoped I managed to make it clear.

[poodel]

Ok, I've got all the hardware now, and I'm trying to get it to work. It's:

Core2Duo E6300
Asus P5-V
2GB DDR2
4 x 750GB Seagate Barracuda 7200.10 (for the RAID5)
2 x 500GB Seagate Barracuda 7200.10 (for the root)
Areca 1220

The latest Debian Testing (2007-01-11) has native support for the Areca array, so that's no problem. I just got the pre-compiled binaries from the Areca FTP site to interact with the card. Works fine with Core2Duo as well.


I'm currently facing some problems with getting Linux to understand when I change the size of the array, or add another volume in the array. The current workaround is to reboot. I guess I'm looking for some rescan-the-SATA-bus-functionality. Any tips on that?

Also not sure how I will do the root RAID1 yet. I've got four possibilitys:
1) RAID1 via the Areca controller. This would work, but I don't want to "waste" two ports.
2) The motherboard has some RAID1 capability via the JMicron controller, but interacting with it doesn't seem to be possible from anywhere else than the BIOS boot menu. How would I know if a drive fails?
3) MD
4) LVM

Any tips on the RAID1 would be appreciated.

[Big Buck Hunter]

Haversian, on Jan 6 2007, 01:41 AM, said:

you're reading 3 stripes, computing, and writing the 4th stripe, for about 25MB/sec throughput. I presume the stripe cache deserves credit for write performance in excess of that figure.

stripe...... cache.......?

about 25MB/sec?

Puts head on desk.

BBH

[Big Buck Hunter]

poodel, on Jan 12 2007, 06:54 PM, said:

I guess I'm looking for some rescan-the-SATA-bus-functionality. Any tips on that?

rmmod and insmod the module for that controller? Keep in mind that this should "not"be the controller you have booted from, or have any live mounts on.

Thank you for your time,
Frank Russo

[poodel]

jpiszcz, on Jan 3 2007, 12:08 PM, said:

Growing the XFS filesystem is a breeze:

# xfs_growfs /raid5

Just thought of one thing... when you add another disk to the set, doesn't that mess with the XFS parameters. Ideally, they would be tuned to your number of hard disks at creation time, but now they won't be?

Is there a way to tune the XFS after creation?

[jpiszcz]

That is a good question.

[SERGEY]

jpiszcz, on Jan 1 2007, 07:26 PM, said:

Get the 750GB ES (for RAID drives) and Areca no question, promise cards are 'fakeraid' - Areca cards are the fastest SATA cards in the world.

Why do you think, say, Promise 8350 is "fakeraid"? It uses the same Intel processor like Areca 1220s IOP333...

[jpiszcz]

SERGEY, on Mar 2 2007, 08:13 AM, said:

jpiszcz, on Jan 1 2007, 07:26 PM, said:

Get the 750GB ES (for RAID drives) and Areca no question, promise cards are 'fakeraid' - Areca cards are the fastest SATA cards in the world.

Why do you think, say, Promise 8350 is "fakeraid"? It uses the same Intel processor like Areca 1220s IOP333...

That one is, but the majority of them are fakeraid.

(link) Promise SuperTrak EX8300 8-port SATA-II PCI-X card and SuperTrak EX8350 8-port SATA-II PCI Express card — real hardware RAID(?). Uses Intel IOP331 I/O processor (XScale family) and Marvell 88SX6081 SATA chipset. Press releases says there's a "full open source Linux driver" — which might be the Marvell driver(?). EX8500 card was demoed on 2005-08-23.

See http://linuxmafia.co...dware/sata.html

Sorry for the confusion/error.