innomind

Fastest hard drive configuration for video editing

Recommended Posts

Hello everyone!

It is my first time building a computer myself and I have couple of questions in regards to choosing what hard drives to get and what RAID set up to go with for blazing fast write speeds. I will be using Adobe video editing and effects programs on this rig so the write speed to hard drives must be extremely fast.

Here is the list of components I will be using to build this machine:

**************************************************************

Intel Core 2 Extreme QX6700 Quad Core Processor

Asus P5WDG2 WS Professional Motherboard

Scythe SCNJ1100P 120mm CPU Cooling Fan

Antec P180 Silver Mid Tower

Corsair CMPSU-620HX 620W ATX EPS12V PS/2 Modular Power Supply

4GB of Kingston 1GB PC2-6400 800MHz 240-pin Non-ECC Unbuffered CL5 DDR2 SDRAM DIMM

PNY NVIDIA Quadro FX 1500 Video Card (MPN: VCQFX1500PCIEPBV)

******************************************************************

So my question is... What is the fastest under 300GB SATAII hard drive on the market today that I can use for c: drive (this drive will not be using any RAID configuration since it is purely for the OS and the apps installed)

So the d: drive will be using RAID0 consisting of two or three hard drives (depending on available capacity of each drive). I heard that WD Raptor hard drives 150GB in capacity are really fast when two of such setup on RAID0 (for WRITE speed performance), is that true? The Asus P5WDG2 WS Professional Motherboard I will use has RAID0 capability, however, I hear going with Areca ARC 12x1 ML PCIe SATAII Raid controller will yield more write speed. Will I gain noticeable speed gain with this controller? Anyone uses this approach?

I shoot a lot of video and I think having the d: drive dedicated for editing video only will be both fast and economical. The finished edited video projects will then be moved to the external USB/Firewire drives for long-term storage.

Any comment coherent to this is really appreciated.

Thanks a lot!

Renat Zarbailov of Innomind.org

Share this post


Link to post
Share on other sites

Hi Renat,

Have a look through the performance database for which drive to use as the OS drive and data drives...

The add-in RAID card can have benefits as it can offload processing from the CPU, but how much that will help in RAID0 is debatable, (as RAID0 has very little processing requirements).

Also what OS are you going to use?

PS. Welcome to SR! :)

Share this post


Link to post
Share on other sites

Hi,

I'd recomend the Coolermaster Stacker over the Antec case. Cooler, more room, quieter.

IMO. Run, don't walk, as fast as possible, away, from any raid on any Asus motherboard!

It's not SATAII but a 150g Raptor for a single drive OS is as good as it get these days. For now.

I would suggest 2 different raid arrays to edit video on, the ability to go back and forth between two super fast drives greatly helps during edits/capturing. 3 500G WD "raid-editions" for each array would be very fast quiet and relativly cool. 2 would also get the job done. Important that whatever you end up with, it is not on the pci bus!

2 Raptors would hit the 150mb speed limit and not very cost effective per gigabyte.

Peter

Share this post


Link to post
Share on other sites
Hello everyone!

It is my first time building a computer myself and I have couple of questions in regards to choosing what hard drives to get and what RAID set up to go with for blazing fast write speeds. I will be using Adobe video editing and effects programs on this rig so the write speed to hard drives must be extremely fast.

Why must the disk subsystem be "extremely fast"? Are you editing raw uncompressed video files, or are you working with a compressed format like DV/MPEG? Which Adobe program will you be using? What resolutions and bitrates of video do you expect to work with?

If the answer to any of the above questions is, "I don't know", I would recommend purchasing a single 500GB drive, running for a while, and figuring out the answer. You can add an appropriate storage subsystem once you have a better understanding of what performance characteristics you require.

Thank you for your time,

Frank Russo

Share this post


Link to post
Share on other sites
What is the fastest under 300GB SATAII hard drive on the market today that I can use for c: drive

The fastest SATA hard drive on the market is WD's Raptor 150, which is not SATA-II. Don't be a marketing tool. If you want faster you'll have to resort to 15K SCSI and even then, the performance benefit for your usage pattern may not be clear.

I heard that WD Raptor hard drives 150GB in capacity are really fast when two of such setup on RAID0 (for WRITE speed performance), is that true? The Asus P5WDG2 WS Professional Motherboard I will use has RAID0 capability, however, I hear going with Areca ARC 12x1 ML PCIe SATAII Raid controller will yield more write speed. Will I gain noticeable speed gain with this controller? Anyone uses this approach?

The Raptors are fast (see first answer). Any drives in RAID-0 will be faster than they are standalone, at the risk of (exponentially?) increased probability of a single failure disabling the entire volume.

An good off-motherboard RAID controller should help performance slightly by offloading CPU cycles, and perhaps even more by the ability to host its own cache - up to 1GB on some controllers. Whether it's really necessary is questionable however. Adobe is pretty good about prefetching for good perceived performance.

-Brad

Edited by Brad.B

Share this post


Link to post
Share on other sites

Peter99: "3 500G WD "raid-editions" for each array would be very fast quiet and relativly cool. ...

2 Raptors would hit the 150mb speed limit and not very cost effective per gigabyte."

1) So is a 2x 3x 500GB arrays (6 500GB drives = 3 terabytes) cost-effective? If they are only used for temporary files and ready works are transfered to another drive as soon as possible, wouldn't there most likely be ~100GB used space on that 3000GB of available capacity? Consider the price of 6x WD5000YS for 100GB of used space... that's a high price per gigabyte. 4x 74GB Raptor would already offer 148+148GB. And with a cost of around 150€ a piece, that'd be 600€ where as two 1.5 terabyte arrays would cost 1000€. So for a 100GB work file that's 6€/GB at Raptors and 10€/GB at 500GB WD arrays.

2) What 150MB speed limit? You mean the SATA150 interface? But those drives will be using different ports (unless he'll be using port multipliers). Each port have their own 150MB/s. (Unlike PATA which support two drives (MASTER & SLAVE) per cable.)

Share this post


Link to post
Share on other sites

Renat, Are you now or planning on shooting High Def?

Where will your virtual memory be located?

What sustained write speed do you desire?

Consider a MbP with a SSD sata HD, then Xfer via 1394b to storage.

What is the longest single project you anticipate ?

(1 hour shoot or 3 full day seminars =24 hours of related topics as 1 project to output to a DVD etc)

Even working with standard NTSC/PAL raw avi 1 hour =13 to 17 GB as I recall (corrections accepted!)

For DVE AVOID raid 0, go RAID 3 or higher [as already recomended 4x HD in RAID 10 (aka 0+1)]

SNPD w/ COMMENTS

building a computer, couple of questions; choosing what hard drives to get, what RAID for blazing fast write speeds.

Adobe video editing and effects programs; write speed to hard drives must be extremely fast.

Here is the list of components I will be using to build this machine:**************************

May seem some benefit in a 2-3 years when Adobe rewrites for 64bit. CPU= Core 2 Extreme QX6700 Quad

NOT SURE why someone said to avoid a Asus raid. mobo=P5WDG2 WS Professional

If serious RAID, may need more guts in PSU=Corsair CMPSU-620HX 620W ATX EPS12V PS/2

Depends on OS, if xpPRO 4x is waste $$, 2x will run better (Asus=problems>2x) RAM=4x 1GB Kingston PC2-6400

Go DUAL DVI, I do not know this card. PNY NVIDIA Quadro FX 1500 Video Card (MPN: VCQFX1500PCIEPBV)

******************************************************************

questions ARE...fastest under 300GB SATAII hard drive on the market today that I can use for c: drive (this drive will not be using any RAID configuration since it is purely for the OS and the apps installed)

d: drive will be using RAID0 consisting of two or three hard drives (depending on available capacity of each drive).

I shoot a lot of video and I think having the d: drive dedicated for editing video {DVE*} only will be both fast and economical. The finished edited video projects will then be moved to the external USB/Firewire drives for long-term storage.Renat Zarbailov of Innomind.org

For your 1st build I would NOT go with bleeding edge specs, = NOT economical! Get a TYAN server mobo, use 4 WD740GD in Raid 10,.

Speaking from experience, as now have 5 DT {AMD 939 nF4} + LT {Pentium 725 on Asus M7V} I built and migrating to GNU/Linux. Moving away from vista ULTIMATE (been testing since Jun 06)

Share this post


Link to post
Share on other sites
1) So is a 2x 3x 500GB arrays (6 500GB drives = 3 terabytes) cost-effective? If they are only used for temporary files and ready works are transfered to another drive as soon as possible, wouldn't there most likely be ~100GB used space on that 3000GB of available capacity? Consider the price of 6x WD5000YS for 100GB of used space... that's a high price per gigabyte. 4x 74GB Raptor would already offer 148+148GB. And with a cost of around 150€ a piece, that'd be 600€ where as two 1.5 terabyte arrays would cost 1000€. So for a 100GB work file that's 6€/GB at Raptors and 10€/GB at 500GB WD arrays.

We don't really know for sure what kind of capture speeds/capacity he needs. But the man did state a desire for 2 or 3 capture drives, So how did you get it to 6? Maybe your right, 2 or 3 250GB WD JD's would probably be fine, if there is not a need to hold more than a couple of day's worth of work. $75*3 +225 = $450 total. Very cost effective.

2) What 150MB speed limit? You mean the SATA150 interface? But those drives will be using different ports (unless he'll be using port multipliers). Each port have their own 150MB/s. (Unlike PATA which support two drives (MASTER & SLAVE) per cable.)

Two Raptor 150's on the Asus DH-Deluxe MB i have will not go over 150 MB/sec STR with any of the various raid configurations available.

Peter

Edited by Peter99

Share this post


Link to post
Share on other sites

Peter99: "But the man did state a desire for 2 or 3 capture drives, So how did you get it to 6?"

How did I get it to 6?

Quoting you: "I would suggest 2 different raid arrays ... 3 500G WD "raid-editions" for each array"

Thus, you said it yourself. If you can't do the math: 2 times 3 is 6.

Share this post


Link to post
Share on other sites

He didn't say capture, he said editing.

I will be using Adobe video editing and effects programs on this rig

Editing compressed video doesn't take much hard drive speed, it does take a lot of CPU power.

Editing uncompressed video takes a lot of hard drive speed, it doesn't take a lot of CPU power.

So yeah, it depends greatly on the user's requirements, which have not yet been stated.

However, even 3 Raptors in RAID0 won't be fast enough for real time uncompressed HD editing.

Share this post


Link to post
Share on other sites

Capture of std avi only requires ~4MB/s I thought, which supposedly is well within the capability of USB or FW (concern is lost frames).

This Would apply/includes transfer of finished file to HD storage? Why then the need for FW800/1394b??

SNPD However, even 3 Raptors in RAID0 won't be fast enough for real time uncompressed HD editing.
TY for an informative/fairly clear post.

But irt RAW avi (aka uncompressed in your meaning, but raw has neither been edited or compressed) I thought rendering in Adobe was limited by drive speeds, while editing raw avi took some CPU guts.

What speed is fast enough for real time uncompressed HD editing? vs Recommended speed for std NTSC?

Capture of HD video to hard drive=?? {Max capture rate required determined by camera Xfer capability??}

Has anyone found a GNU/Linux distribution and non-linear edit package that is in same league as Adobe P, or FC, Pro?? Do the same MB/s guidelines still apply from MS Win / Adobe, Apple/ FCP? TY all, d8d

Share this post


Link to post
Share on other sites
Hello everyone!

It is my first time building a computer myself and I have couple of questions in regards to choosing what hard drives to get and what RAID set up to go with for blazing fast write speeds. I will be using Adobe video editing and effects programs on this rig so the write speed to hard drives must be extremely fast.

Here is the list of components I will be using to build this machine:

**************************************************************

Intel Core 2 Extreme QX6700 Quad Core Processor

Asus P5WDG2 WS Professional Motherboard

If you want to see what your QX6700 is capable of with respect to video encoding you should check this out...

http://www.xtremesystems.org/forums/showthread.php?t=126594

As for drives... He's using dual Areca cards each with 4x 76GB 16MB Raptors in RAID 0... and he is disk limited! :)

Edited by virtualrain

Share this post


Link to post
Share on other sites
If you want to see what your QX6700 is capable of with respect to video encoding you should check this out...

http://www.xtremesystems.org/forums/showthread.php?t=126594

As for drives... He's using dual Areca cards each with 4x 76GB 16MB Raptors in RAID 0... and he is disk limited! :)

I call bullshit on this.

He is just doing a mpeg2 requantisation. _real_ encoding will be about an order of magnitued slower, take h264 and HD and you have 2 orders of magnitude.

Share this post


Link to post
Share on other sites
Capture of std avi only requires ~4MB/s I thought, which supposedly is well within the capability of USB or FW (concern is lost frames).

This Would apply/includes transfer of finished file to HD storage? Why then the need for FW800/1394b??

SNPD However, even 3 Raptors in RAID0 won't be fast enough for real time uncompressed HD editing.
TY for an informative/fairly clear post.

But irt RAW avi (aka uncompressed in your meaning, but raw has neither been edited or compressed) I thought rendering in Adobe was limited by drive speeds, while editing raw avi took some CPU guts.

What speed is fast enough for real time uncompressed HD editing? vs Recommended speed for std NTSC?

Capture of HD video to hard drive=?? {Max capture rate required determined by camera Xfer capability??}

Has anyone found a GNU/Linux distribution and non-linear edit package that is in same league as Adobe P, or FC, Pro?? Do the same MB/s guidelines still apply from MS Win / Adobe, Apple/ FCP? TY all, d8d

If your working in DV, you stay in DV format till you wish to render. Usually, people render straight to MPEG2 for DVD distribution, or MPEG4 for internet distribution. At no point is the video converted into raw format.

In Linux, I use kino and cinerella. For legacy operating systems, I used to use Vegas.

Thank you for your time,

Frank Russo

Share this post


Link to post
Share on other sites

If you want to see what your QX6700 is capable of with respect to video encoding you should check this out...

http://www.xtremesystems.org/forums/showthread.php?t=126594

As for drives... He's using dual Areca cards each with 4x 76GB 16MB Raptors in RAID 0... and he is disk limited! :)

I call bullshit on this.

He is just doing a mpeg2 requantisation. _real_ encoding will be about an order of magnitued slower, take h264 and HD and you have 2 orders of magnitude.

At any rate, his disk setup is amazing and he claims it's still holding him back.

Edited by virtualrain

Share this post


Link to post
Share on other sites

BBH, TY for GNU/Linux info! btw, My $500 RAID systems >100MB/s read, but do not recall exact write (~60?). Will BM again w/ full details. And I only have as single core AMD FX55 that cost me $199retail.

Do kino & cinerella have similar capabilities to APP, FCP? When I saw kino in use it did not appear to be real time non-linear edit. May have been because the user was new to setting it up?

Capture of std avi is well within the capability of USB or FW (concern is lost frames??).

Would this apply/includes transfer of finished file to HD storage?

Why then the need for FW800/1394b that Apple sells (required only for HDTV ?)?

But irt RAW avi (aka uncompressed in your meaning, but raw has neither been edited or compressed) I thought rendering in Adobe was limited by drive speeds, while editing raw avi took some CPU guts.

Do the same MB/s guidelines for non-linear edit still apply from MS Win / Adobe, Apple/ FCP? TY all, d8d

By raw avi I meant what is captured from miniDV tape, which I think = you are calling DV format. Not sure what was meant by RT uncompressed HD editing
If your working in DV, you stay in DV format till you wish to render. Usually, people render straight to MPEG2 for DVD distribution, or MPEG4 for internet distribution. At no point is the video converted into raw format.

In Linux, I use kino and cinerella. For legacy operating systems, (D8D=???) I used to use Vegas.

Is there any way to edit your own posts in this forum?? Have option ASAP after post, but when reenter=NO edit option displayed.

Share this post


Link to post
Share on other sites

No, I think we misunderstand each other. Raw=uncompressed. DV=DV. You cap in DV, you edit in DV, and you render in any format you like. At no point does the video get uncompressed. You are always working at 3.6MB/sec, until you do a transition, at which point you are working at 7.2MB/sec. At the end of the transition (star wipe rocks!!), you are back to 3.6 MB/sec.

Kino=MS Movie equiv. I use it for linear editing and capture. I probably use it cause I use KDE. I'm sure gnome has the same thing under a different name (like Gino or something).

Cinerella=Vegas equiv, without a lot of the features I liked in Vegas. (I'm really a vegas guy, but can't run it on my distro at the moment)

For crap work, I use avidemux or virtualdub under wine. They're linear, but have the filters I need to do digitized VHS fixups.

You can edit posts for about 5 minutes after the post, at which point the post is locked. It prevents a lot of "Uh-huh, Nut-uh" arguments, and allows us to look back at the dumb things we've all said over the years.

Frank

PS, if you use windows and want to try vegas, grab the demo here.

http://www.sonymediasoftware.com/products/...uct.asp?pid=404

Edited by Big Buck Hunter

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now