The Belgain

3-drive Linux software RAID5 performance

Recommended Posts

Hi there,

I've just set up a Linux sofware RAID5 array, and was wondering whether the performance numbers I'm seeing are about right or a bit low? I had two 320GB drives and 2 160GB drives so I've RAID0'ed the 160s and created a "3-drive" raid5 from that and the 320s. The 320s are WDs, and the 160s are Maxtor DM9s (all SATA). They're running on the onboard ICH6 SATA controller, CPU is a 2.4GHz P4.

Bonnie gives the following:

james@ubuntu-fileserver:~$ bonnie -d /home/james/raid5array/bonnie
Writing with putc()...done
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.03	   ------Sequential Output------ --Sequential Input- --Random-
				-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine		Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
ubuntu-fileserve 1G 29959  82 54596  11 21789   7 28181  73 73167  15 281.6   1
				------Sequential Create------ --------Random Create--------
				-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
		  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
			 16  2687  96 +++++ +++ +++++ +++  2798  97 +++++ +++  8841  99
ubuntu-fileserver,1G,29959,82,54596,11,21789,7,28181,73,73167,15,281.6,1,16,2687,96,+++++,+++,+++++,+++,2798,97,+++++,+++,8841,99

Share this post


Link to post
Share on other sites

Yeah I was pretty puzzled about that too, but is this just bonnie being a little confusing about it's output? Is it talking about input and output to the program rather than the drive (so that the left figures are writes and the right figures are reads)?

Yes I've run it multiple times, on an idle system. The results are very consistent between runs.

Share this post


Link to post
Share on other sites
Yeah I was pretty puzzled about that too, but is this just bonnie being a little confusing about it's output? Is it talking about input and output to the program rather than the drive (so that the left figures are writes and the right figures are reads)?

Yes I've run it multiple times, on an idle system. The results are very consistent between runs.

That's due to buffering.

Try: bonnie -s 4000 -d /home/james/raid5array/bonnie

Unless you have 4GB of RAM, you should see remarkably slower write results.

Share this post


Link to post
Share on other sites

Here's what I got from my "top" system, NOT meant to be a direct comparison, since there's nothing similar in HW/SW configuration. This is just for reference or folks' curiosity :D

Sun W2100z 2xOperton 246 (2GHz), 2GB memory.

Adaptec ASC-29320A U320

RAID-Z of 3 disks: FUJITSU-MAU3036NP-0104-34.25GB x 2 + SEAGATE-ST336753LW-0003-34.18GB

RAID-Z is introduced with ZFS in OpenSolaris.

If you're not familiar with it, like how it compares to RAID-5, Jeff Bonwick's Weblog is a good read.

$ uname -a

SunOS w2100z 5.11 snv_27 i86pc i386 i86pc

bonnie++ Version 1.01

(Downloaded from http://www.textuality.com/bonnie/download.html)

$ bonnie++ -s 4096M -d /tank

Version  1.01	   ------Sequential Output------ --Sequential Input- --Random-
				-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine		Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
w2100z		   4G 54268  61 50938  16 69013  21 77427  95 152428  23 502.5   1
				------Sequential Create------ --------Random Create--------
				-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
		  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
			 16 31461  97 +++++ +++ 22759  59 30883  99 +++++ +++ 27121  99
w2100z,4G,54268,61,50938,16,69013,21,77427,95,152428,23,502.5,1,16,31461,97,+++++,+++,22759,59,30883,99,+++++,+++,27121,99

Share this post


Link to post
Share on other sites

I have a linux software raid5 with 4 Samsung SP1614C (160 GB SATA) on a SI-3114. Machine is a AMD64 3000+ on a Soltek VIA chipset 939 board. Here are my results:

root@livingroom:/home/yuchai# bonnie++ -s 4048 -d /home/yuchai/iso -u yuchai -g users
Using uid:1000, gid:100.
Writing with putc()...done
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.03	   ------Sequential Output------ --Sequential Input- --Random-
				-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine		Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
livingroom	4048M 28899  71 50786  18 26024  14 38307  85 61047  15 324.0   1
				------Sequential Create------ --------Random Create--------
				-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
		  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
			 16  2476  96 +++++ +++ +++++ +++  2730  98 +++++ +++ 10552  99
livingroom,4048M,28899,71,50786,18,26024,14,38307,85,61047,15,324.0,1,16,2476,96,+++++,+++,+++++,+++,2730,98,+++++,+++,10552,99

Edited by yuchai

Share this post


Link to post
Share on other sites

Well, I've added the 4th drive to my array now, and the results I'm seeing are below. This is now a RAID5 array of 4 320GB WD SATA drives.

james@ubuntu-fileserver:~$ bonnie -d raid5array/bonnie/
Writing with putc()...done
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.03	   ------Sequential Output------ --Sequential Input- --Random-
				-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine		Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
ubuntu-fileserve 1G 29971  83 58920  13 20682   6 26587  70 86801  17 259.6   0
				------Sequential Create------ --------Random Create--------
				-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
		  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
			 16  2815  97 +++++ +++ +++++ +++  2935  98 +++++ +++  9034  99
ubuntu-fileserver,1G,29971,83,58920,13,20682,6,26587,70,86801,17,259.6,0,16,2815,97,+++++,+++,+++++,+++,2935,98,+++++,+++,9034,99

I'll rerun with the -s parameter to see if the results differ very much.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now