TheR

Promise SuperTrak EX8350 PCIe x4

Recommended Posts

Hey everybody :]

A few days ago I finally got to install my new EX8350 in my also brand new gentoo Box and since there's a lot of discussion going on about the performance, I did some benchmarking.

System Details:

Dualcore Opteron 165, 2.7 GHz (slightly overclocked from 1.8 GHz) with 2 GiB of DDR-SDRam on an ASUS A8N32-SLI (note: You cannot install the EX8350 in the PCIe x4 slot on this mobo since it's too large (actually the heatpipe's positioned wrong, blaming Promise for that isn't nice.), so it's in the second x16 slot.), an EX8350 with 8 Western Digital RE2 Disks (320G each).

Benchmarking was done on a RAID-5 Volume almost exactly 2.0 TiB large. The remaining 40 or so gigs are currently neither used nor mounted. Driver and BIOS version is 2.9.10.

The volume contains one partition which occupies all of the volume's size and is formatted with xfs.

I ran dd with different block sizes copying 1.0 GBytes back and forth from the disk volume to a ramdrive. Here are the results, sorted by dd's bs (BlockSize) argument. Each test was run three times to avoid false results.

Reading 1.0 GByte of data from /mnt/sda1 (raid volume) to /dev/shm (ramdrive)

BS 4M: 448 MB/s, 441 MB/s, 449 MB/s

BS 2M: 452 MB/s, 443 MB/s, 453 MB/s

BS 1M: 444 MB/s, 449 MB/s, 444 MB/s

BS 512k: 522 MB/s, 508 MB/s, 528 MB/s

BS 256k: 697 MB/s, 667 MB/s, 699 MB/s

BS 128k: 708 MB/s, 692 MB/s, 706 MB/s

BS 64k: 683 MB/s, 701 MB/s, 690 MB/s

BS 32k: 713 MB/s, 688 MB/s, 714 MB/s

BS 16k: 679 MB/s, 692 MB/s, 680 MB/s

BS 8k: 627 MB/s, 652 MB/s, 630 MB/s

Writing 1.0 GByte of data from /dev/shm (ramdrive) to /mnt/sda1 (raid volume)

BS 4M: 159 MB/s, 315 MB/s, 344 MB/s

BS 2M: 302 MB/s, 313 MB/s, 332 MB/s

BS 1M: 344 MB/s, 306 MB/s, 294 MB/s

BS 512k: 307 MB/s, 131 MB/s, 302 MB/s

BS 256k: 315 MB/s, 316 MB/s, 325 MB/s

BS 128k: 336 MB/s, 317 MB/s, 325 MB/s

BS 64k: 182 MB/s, 319 MB/s, 388 MB/s

BS 32k: 269 MB/s, 321 MB/s, 319 MB/s

BS 16k: 321 MB/s, 310 MB/s, 313 MB/s

BS 8k: 312 MB/s, 310 MB/s, 275 MB/s

Those results really surprised me - especially the write performance differences between different Block sizes; from 445 MB/s with 4M to about 700 MB/s with 32k is a real difference in performance. Earlier I have conducted some tests in windows (didn't save any of them though) and got read speeds of about 500 MB/s over the whole disk, so I guess the linux driver isn't too bad after all :]

The write rates were a bit more even balanced, except for a few really slow transfers (159 MB/s, 131 MB/s...) and one surprisingly fast (388 MB/s) all of them were in the range of 300 MB/s +-50 MB/s, so I assume either xfs or the controller catch those block sizes and reformat them nicely.

Just for the sake of checking if read rates of /dev/shm are fast enough I did some test there, copying stuff to /dev/null (i.e. nowhere). Results are as follows:

BS 4M: ~960 MB/s

BS 2M: ~968 MB/s

BS 1M: ~967 MB/s

BS 512k: 1.3 GB/s

BS 8-256k: 2.0-2.2 GB/s (one out of 18 runs resulted in 1.9 GB/s, all of the other ones between 2.0 and 2.2 GB/s)

I hope those results are useful to someone, I like the card, it runs stable and everything apprears to be fine here. I would however not recommend it to windows users (let's make that a "I wouldn't recommend running windows on a box with that kind of storage"), it might run, but it might as well not run either.

Share this post


Link to post
Share on other sites

I also got a EX8350 card mounted in a PCI-16x slot running W2k3 and I don't get those high speed. I get around 133 mb/s in avg speed (messured by HDtach) on a 5 x seagate 7200.10, 320 GB drives with onboard cache set to Write back. So don't know if the driver is purly optimized under W2k3 ?

a second question is, how do you set the block size (I think you mean strip size) to other values then 32, 64 and 128 kb ??

Henrik.

Share this post


Link to post
Share on other sites

Henrik:

I had the same setup (volume size, back then it was formatted with NTFS) running on '03 and got speeds of 450-500 MB/s in HDTach, so I guess the WDs are just plain faster than your Seagates. Plus, I've got 8, you only 5.

The answer to the second question is quite easy, I am talking about dd's "bs" option, not the stripe size. The bs just changes how many bytes are read and written at a time (i.e. bs=512k would read 512k of data, write 512k, read the next 512k and so on). Iirc you can't change the stripe size "on the fly" but would have to rebuild the array after every change. And seriously, I don't have that kind of time to spend :P

Share this post


Link to post
Share on other sites
I ran dd with different block sizes copying 1.0 GBytes back and forth from the disk volume to a ramdrive.

You have a 1 GB test file and 2 GiB RAM. I have a different system, but it can certainly cache a 1 GB file entirely in 2 GiB RAM.

Do you have any sample results with e.g. a 10 GB file using /dev/null ?

Share this post


Link to post
Share on other sites

I ran dd with different block sizes copying 1.0 GBytes back and forth from the disk volume to a ramdrive.

You have a 1 GB test file and 2 GiB RAM. I have a different system, but it can certainly cache a 1 GB file entirely in 2 GiB RAM.

Do you have any sample results with e.g. a 10 GB file using /dev/null ?

Or better yet, bonnie++ is a simple device/filesystem test program that beats caching. I was able to reach around 200mb/s with the driver downloaded from promise.com (i don't know if the newer driver scheduled to go into the main linux kernel soon is faster) with 7 maxtor maxline iii drives in raid 5, with a opteron 165 running at stock speed.

Share this post


Link to post
Share on other sites

It's been quite some time since I've last checked this thread, sorry about letting you wait for my reply.

I have ran bonnie++ and gotten quite... weird results, but look for yourself:

Params:  -s 4096 (ram*2) -r 2048

Version	 1.93c   ------Sequential Output------ --Sequential Input- --Random-
				-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine		Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
localhost		4G   743  99 45234  10 31568   7   876  77 407506  54 235.7   3
Latency			 12187us	 918ms	 893ms	 789ms   60838us	1093ms
				------Sequential Create------ --------Random Create--------
				-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min		/sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
localhost		16  2639  13 +++++ +++  3534  23  3348  21 +++++ +++  2708  14
Latency			  1480ms	  32us	 852ms	 969ms	  33us	 772ms

What really confuses me is the Seq. Out-/Input Block speeds; apparently writing is possible up to 400 MByte/s but reading only up to about 45 MByte/s. I got similar results when copying big files (like 10G) from and to /dev/null.

Has anyone of you guys got an idea what could be wrong or how I could fix it, maybe?

Edited by darthflo

Share this post


Link to post
Share on other sites

Just plugged 5 WD5000YS drives into this for a RAID 5 set up.

BS 128

Write Through

and 512 sections

ran ATTO and HDTECH -> preliminary results seem to show read speeds averaging around 150MB/s with Burst around 200 MB/s

write speeds are not that good.

The best I have seen is around 70MB/s for larger files

Share this post


Link to post
Share on other sites

OK, so after further testing,

HDtach is reporting a fairly steady read speed of avg 60 MB/s for read all throughout the array.

Burst has gone down to 160

Atto is showing that small files get the short end of the stick, but 128 kb files clear the 50 MB/s write speed and after that it looks good.

Read speeds are much better, around the 4kb mark they surpass 73MB/s

the 1024 kb transfer of a 32mb file shows a nice 140 MB/s write speed

and 1024 and 512 both seemed to have maxed out the read capabilities reaching 233 MB/s

works for me :)

especially since i have a lot of movies and anime of my storage drives.

Share this post


Link to post
Share on other sites

I just want to inform you that driver for EX8350 is included in Ubuntu 7.04. The card was recognized and installed automaticaly and I was able to access 2T partition instantly.

It looks like driver is now part of Linux kernel. Festy is based on 2.6.20.

by

TheR

Share this post


Link to post
Share on other sites
Hello All - 1st post :)

After installing an EX8350 on a MSI K8N Neo4 MB w/4x Seagate 120G SATA drives

When I boot the system the card identifies itself SuperTrak EX8350/8300 BIOS version 2.8.0.25

Scanning disks...... about 250 dots - takes about 3 minutes

Warning - Something wrong with your hardware!.

The system hangs here.

I have rebooted several times with and without the drives connected to the card - all 4, 2 drives, 1 drive - the result is the same every time - it takes 2-3 minutes in the Scanning Disks mode then fails

Is there something wrong with my card or the drives

what can I try next.

Please Help!!!

Thank you,

Will :)

I have a similar issue with my installation, using a MSI p965Neo Motherboard, 6 WD400 SATA II drives, and the Promise SuperTrak EX8350.

When I first boot my system, everything seems fine. I do the control - S thing to hop into the configuration of the EX8350, and I can see all 6 drives. I Then attempt to build a raid 5 array, and after a few minutes, two of my drives drop off and the array goes critical.

After I reboot the server, I get an error about the previous attempt over clock setting caused a problem. I reset the system setting to default, and after another reboot, I can see the two dropped drives again. At first, I thought I had a couple of bad drives, so I returned them and got two new ones. That didn't solve the issue. Then I thought that the drive trays were causing the issue, (I have a 6 tray hot swap bay enclosure) so I removed the bay and attached all the drives directly to the card, but the same issue occurs.

I have moved the drives to different channels to no avail either.

Then I tried creating an array with only 4 drives. This array seemed to work at first, but after a little while, one of the drives disappeared again.

I should only need about 5ooW of power to run this system, and I have a 600w power supply.

I'm in one hell of a fix right now. This system that I am building will server as an ISCSI server, hosting disk space to a file, SQL, and application clusters.

Any ideas? I'll buy a beer for anyone that can help me figure this thing out.

Share this post


Link to post
Share on other sites
Hello All - 1st post :)

After installing an EX8350 on a MSI K8N Neo4 MB w/4x Seagate 120G SATA drives

When I boot the system the card identifies itself SuperTrak EX8350/8300 BIOS version 2.8.0.25

Scanning disks...... about 250 dots - takes about 3 minutes

Warning - Something wrong with your hardware!.

The system hangs here.

I have rebooted several times with and without the drives connected to the card - all 4, 2 drives, 1 drive - the result is the same every time - it takes 2-3 minutes in the Scanning Disks mode then fails

Is there something wrong with my card or the drives

what can I try next.

Please Help!!!

Thank you,

Will :)

I have a similar issue with my installation, using a MSI p965Neo Motherboard, 6 WD400 SATA II drives, and the Promise SuperTrak EX8350.

When I first boot my system, everything seems fine. I do the control - S thing to hop into the configuration of the EX8350, and I can see all 6 drives. I Then attempt to build a raid 5 array, and after a few minutes, two of my drives drop off and the array goes critical.

After I reboot the server, I get an error about the previous attempt over clock setting caused a problem. I reset the system setting to default, and after another reboot, I can see the two dropped drives again. At first, I thought I had a couple of bad drives, so I returned them and got two new ones. That didn't solve the issue. Then I thought that the drive trays were causing the issue, (I have a 6 tray hot swap bay enclosure) so I removed the bay and attached all the drives directly to the card, but the same issue occurs.

I have moved the drives to different channels to no avail either.

Then I tried creating an array with only 4 drives. This array seemed to work at first, but after a little while, one of the drives disappeared again.

I should only need about 5ooW of power to run this system, and I have a 600w power supply.

I'm in one hell of a fix right now. This system that I am building will server as an ISCSI server, hosting disk space to a file, SQL, and application clusters.

Any ideas? I'll buy a beer for anyone that can help me figure this thing out.

----------------------------------------------------

I am seeing it too. Any new info?

Share this post


Link to post
Share on other sites
Hello All - 1st post :)

After installing an EX8350 on a MSI K8N Neo4 MB w/4x Seagate 120G SATA drives

When I boot the system the card identifies itself SuperTrak EX8350/8300 BIOS version 2.8.0.25

Scanning disks...... about 250 dots - takes about 3 minutes

Warning - Something wrong with your hardware!.

The system hangs here.

I have rebooted several times with and without the drives connected to the card - all 4, 2 drives, 1 drive - the result is the same every time - it takes 2-3 minutes in the Scanning Disks mode then fails

Is there something wrong with my card or the drives

what can I try next.

Please Help!!!

Thank you,

Will :)

I have a similar issue with my installation, using a MSI p965Neo Motherboard, 6 WD400 SATA II drives, and the Promise SuperTrak EX8350.

When I first boot my system, everything seems fine. I do the control - S thing to hop into the configuration of the EX8350, and I can see all 6 drives. I Then attempt to build a raid 5 array, and after a few minutes, two of my drives drop off and the array goes critical.

After I reboot the server, I get an error about the previous attempt over clock setting caused a problem. I reset the system setting to default, and after another reboot, I can see the two dropped drives again. At first, I thought I had a couple of bad drives, so I returned them and got two new ones. That didn't solve the issue. Then I thought that the drive trays were causing the issue, (I have a 6 tray hot swap bay enclosure) so I removed the bay and attached all the drives directly to the card, but the same issue occurs.

I have moved the drives to different channels to no avail either.

Then I tried creating an array with only 4 drives. This array seemed to work at first, but after a little while, one of the drives disappeared again.

I should only need about 5ooW of power to run this system, and I have a 600w power supply.

I'm in one hell of a fix right now. This system that I am building will server as an ISCSI server, hosting disk space to a file, SQL, and application clusters.

Any ideas? I'll buy a beer for anyone that can help me figure this thing out.

----------------------------------------------------

I am seeing it too. Any new info?

Update: Yes I was able to resolve this issue, although I don't know how helpful this will be to you. I purchased my first 4 drives and 2 days later purchased two additional drives.

The two drives that I purchased came from a defective lot, and when I had them replaced, that second set also came from that same lot. When I contacted the hard drive manufacturer they were able to look up the serial number and confirm that all 4 drives had come from a bad lot. The third set of replacements were perfectly fine.

I wish I had a better answer than that. But for the past 6 months, my ISCSI server has been running rock solid.

Good luck,

Chris

Share this post


Link to post
Share on other sites

Hey everyone. First post :D

I have the Promise ex8350 and have had a positive experience with it.

I have two RAID's on it at the moment.

- a boot RAID-0 made of 2 x WD Raptor 74gb

- a data RAID-5 made of 6 x Seagate 320gb 7200.10

I have used it with the following O/S's without problems:

- XP Pro x86 SP2

- XP Pro x86 SP3 beta

- XP Pro x64 SP2

- Vista x86 (without SP1...before SP1 was out)

- Vista x64 SP1

I have some benchmark numbers. This was on my RAID-5 array and under XP Pro SP2 at the time. The RAID-5 is partitioned as a single 1.6 TB logical disk, is formatted NTFS, and uses a 64k stripe. I used a free benchmark program called "DiskBench" (found here: http://www.nodesoft.com/DiskBench/Default.aspx).

***********************************************************************

Starting Create File Bench...

Created file: D:\Temp\DiskBench1.bin

Size: 1073741824 bytes

Time: 8453 ms

Transfer Rate: 121.140 MB/s

Create File Bench ended

***********************************************************************

Read File Bench started...

Read file: D:\Temp\DiskBench1.bin

Size: 1073741824 bytes

Time: 1109 ms

Transfer Rate: 923.354 MB/s

Read File Bench ended

***********************************************************************

Starting Batch Create File Bench...

256 MB; D:\Temp\batch.bin; 268435456 bytes; 1625 ms; 157.538 MB/s

512 MB; D:\Temp\batch.bin; 536870912 bytes; 3391 ms; 150.988 MB/s

768 MB; D:\Temp\batch.bin; 805306368 bytes; 5109 ms; 150.323 MB/s

1 GB; D:\Temp\batch.bin; 1073741824 bytes; 6906 ms; 148.277 MB/s

Create Batch File Bench ended

***********************************************************************

I also wanted to say thanks to all who posted in this thread as the info given was part of my purchase decision.

CFF.

My complete system specs:

MB - ASUS P5K-E WiFi (Intel P35 chipset, BIOS 1004)

CPU - Core2 Duo e6750 (overclocked to 4.0ghz, 2000mhz FSB)

COOLER - Thermalright Ultra-120 Extreme (with 120mm fan)

VIDEO - EVGA 8800GT 512mb Superclocked (650mhz GPU, 1900mhz RAM, 1620mhz shaders)

RAM - 4gb (4 x 1gb) Transcend aXeRAM DDR2-800 (1000mhz, 5-5-5-15, 2.0v)

P/S - Antec NeoPower 650w

RAID - Promise SuperTrak EX8350 (PCI-e 4x, 8 port SATA II, hardware RAID-6)

HDs - 6 x Seagate 320gb 7200.10 in RAID-5 and 2 x 74gb Raptors in RAID-0

TV - Hauppauge HVR-1800 (PCI-e 1x, analog/digital tuners)

NIC - onboard wireless

SOUND - Soundblaster X-Fi Extreme Audio (PCI-e 1x)

CASE - Antec P182 with 4 x 120mm fans

DVD - SATA LG DVD Burner (GSA-H62N)

FLOPPY - Standard floppy

KBD - Logitech G15 Gaming Keyboard

MOUSE - Logitech G5 Laser Mouse

O/S - XP Pro x86 SP2, Vista Ultimate x64 SP1, XP Pro x86 SP3 BETA

Share this post


Link to post
Share on other sites

Actually those benchmark numbers are old. They were taken when I was using a different motherboard (the ASUS P5N-T Deluxe, nvidia 780i chipset). It unfortunately is quite a P.O.S.

Here are the benchmark numbers I get with my current motherboard (the ASUS P5K-E WiFi):

***************************************

Read file: D:\Temp\DiskBench1.bin

Size: 1073741824 bytes

Time: 750 ms

Transfer Rate: 1365.333 MB/s

Read File Bench ended

***************************************

Created file: D:\Temp\DiskBench1.bin

Size: 1073741824 bytes

Time: 7244 ms

Transfer Rate: 141.011 MB/s

Create File Bench ended

***************************************

They are a little better.

CFF.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now