agnus

Photoshop 2gb+ Files, Need Advice

Recommended Posts

Hi guys! I'm new around here, my english is limited so i'll be brief as much as i can :-)

I work in game development company. My work is to create big, good looking landscapes for wide range of games - simulations, RTS games and so. I work mainly in Photoshop, editing large PSD files, they're approx. 2GB in size (on disk). My company planes to upgrade my workstation, and i have quite a freehand what i get. I decided to run Nforce2 system with Bartie XP-M 2500+ (medium O/C in mind), 1,5GB RAM, Radeon9700... That's for sure. But my problem is when i come to disk system.. I was fascinated by opportunity to play with pair of striped Raptors 36GB... but after reading forum and FAQ... i'm confused.. not to say - in great sorrow ;) So my question is - What can you recommend for me:

1) 2x Striped Raptors 36GB + my 2 old, 5400 drives for storage

or

2) 2x Raptors functioning separately and partitioned for optimum performance (as in many topics, many wise people said?)

or

3) other drives configuration, i can spend max 250$ for HD system.\

Remember that my work depends how fast i'll be able to edit such large file, write it (frequently), close it after 1-2 hour work, open another such big file and edit, write.... .. .. . ..

--

Thanks for advises :-)

Share this post


Link to post
Share on other sites

Congratulations, you are one of the very few who will benefit greatly from RAID-0!

A pair of 36GB Raptors in RAID 0 will do very nicely. But another thing to keep in mind is that your file WILL be more than the RAM in your system can fit. I would highly reccomend trying to get enough RAM in there to fit the OS+Photoshop+the image. Otherwise you will be using virtual memory (page file) all the time. If you have no choice, it would be a very good idea to have another, seperate disk to store your page file. It is very important that this disk be fast. It may actually be cheaper to get more RAM than to buy a 3rd fast drive. If the file is 2GB, than 3GB (3x 1GB sticks) sounds like a really good idea. Make sure your motherboard can support it.

Share this post


Link to post
Share on other sites

Thanks for answering :)

The problem is dual in nature: I'm limited in cost of the whole thing and limited also in fact that NForce motherboards can support up to 3GB memory and they have 3 DIMMs only. The only sticks i can get are 512MB ones.. so.. it means 3x512=1,5GB of RAM. Maybe i'll try to upgrade in the future when 1GB modules becomes more popular.. but as for now i'm stuck with 1,5GBmax :/

The whole machine i have in mind have:

- Asus A7N8X-E-Deluxe

- AXP-M Barton 2500+

- 3x512MB TwinMos DS 2,5CL

- Hercules Radeon 9700 128MB (Ati OEM)

- 2x WD 360GD Raptor

- Chieftec BX case (mieoww! :> )

I have my old drives - 2x 20GB 5400rpm (Seagate & WD) for storage.. i use one of them for PS's scratch... but i'm afraid that in time running Raptor's system - these oldiest would be too slow and become bottleneck for the whole machine, what you think?

--

regards

Share this post


Link to post
Share on other sites

Well, I really love that board (used quite a few myself), but I need to reccomend something with more DIMM slots if you can't get the 1GB sticks.

Another thing to consider is whether PS would be faster using 2 raptors seperatly, one as a dedicated scratch disk...

Share this post


Link to post
Share on other sites

Well, after some more thought, I've reconsidered my last post.

Ditch the second raptor, even change to a standard Barton 2500+, just get the 1GB DIMMS...3GB worth. Really, it's that important.

Crucial will ship these wherever you are.

I know spending $800+ on RAM wan't part of your plan...but jeez, planning on paging is a really bad idea.

Share this post


Link to post
Share on other sites

The problem is that Nforce2 mobos theoretically can't support more than 3 DIMM slots. Gigabyte made such interesting board with four dimmslots but using 4 dimms at once is an issue (single sided, double sided... 1GB as far as i know are all DS so... this exluding Gigabyte's mobo). Summarising - Asus mobo and 3 dimms. I can't get 3x 1GB, it's simply out of my $ range. so i decided to aim 512MB modules and fast hard drive system..

2x Raptors 3GB - i think - are the best solution for me. Resonably cheap and fast... i'm afraid about their noise level (two riding at once??) but i'm determined to get max performance.. i work with headphones all the time too... so... don't matter. I'll play with these babies in single mode and striped... and then i'll decide... i think.

If i strip them - maybe the best solution is to buy third drive specially for PS scratch? $, $, $.......

Share this post


Link to post
Share on other sites

****** 2x Raptors 36GB i had in mind - ofcourse :) 3x 1GB DIMMs is out my dreams...

Share this post


Link to post
Share on other sites
If i strip them - maybe the best solution is to buy third drive specially for PS scratch? $, $, $.......

Well, this sounds like a good "plan B"

I'll play with these babies in single mode and striped... and then i'll decide... i think.

...and this a good "plan C"

i'm afraid about their noise level (two riding at once??)

Don't be too worried, I had 2 in RAID-0 for quite a while, my "quiet" power supply was louder.

Share this post


Link to post
Share on other sites

If you go to a striped pair of Raptors, put only the PS scratch on it, and move the windows pagefile to another disk... perform a search here on the forum, there was a discussion on that very same topic.

Remember to NEVER store important data on a RAID0 !

What version of Photoshop are you running? be careful if you use an earlier version: it will allow you to save a file bigger than 2 gigs, but won't allow you to open it!

PS CS will allow you to use another file format, PSB that has almost no file limit (the 30.000 pixels barrier is gone too, now the limit is 300.000 pixels, with more alpha channels.) In PS CS, the Tiff files can now make up to 4Gigs of ram.

Share this post


Link to post
Share on other sites

--->ddrueding: Thanks for advises... now i have complete plans of A, B and C :)

--->Spectre: I know that risk of "something-going-wrong" with raided disks is bigger than if i would use one disk or two not-paired.. but if this risk is much increased? If i buy two Raptors i'll run out of funds for disk drives.. so maybe it is better for me not to stripe them and leave them working separately?

I use to work on PS 7... and i save my files in PSD format ofcourse. 3D engine use BMP and PNG files.. but they are cut into pieces so it doesn't matter. I know i should go for as much ram for such files as possible.. but i'm forced to have fun with 1-1,5GB only - at the start. Maybe i will obtain 1GB module and then add next modules.. (ok, ok.. i;m getting off topic ;) )

I didn't know that such limitation exists! I'll tell the guys from my company ;) We have really big scapes.. and various versions of PS too. I'm not sure but we have 6th and 7th versions of PS here only.

I'll check the forum in order to find this topic.. if you know exact link.. i'll apreaciate this.. :)

--

best regards to all,

agnus

Share this post


Link to post
Share on other sites

I also have the task of t sorking with the large PS files. First of all, PS (Windows) only sees 2GB. As ylu knows, the scratch disk is usualy teh limiting speed factor. I tried the WDG740GD and found it poor for hte scratch disk compared to a 15k.3. Maybe a stripped pair woudl be a littel beter, but the access time or command queing or something is important. A pair of the 10k WD drives would be fine for storng the PS files, however. By far, the best perfpormance results were obtaned by installing 4GB of RAM and assigng a 2GB RAM scratchdisk. The STR of the RAMDisk is unbeatable at more than 1GB/sec., and the access times are minsucle. Its the biggest speed improvement I’ve had in yeras. :D

But really, realy, if this is work related, buy a Mac with 64-bit supprot up to 8GB. Windows PS is a BS kludge until windows reaches 64-bit status. Also you should conider a dual-CPU system. Now that my harddrives ar not a bottleneck., I wish I had more CPUs.

B. (sorry for the poorwriting)

Share this post


Link to post
Share on other sites

If you have a 500MB image with multiple layers and use over 5 levels of "undo's", take the seperate scratch disk approach.

If you have a large 2GB image, 'try' out the RAID 0 approach. Make certain that the array is properly configured (requires a bit of testing).

Thank you for your time,

Frank Russo

Share this post


Link to post
Share on other sites

If you can't fit everything in RAM the performance is going to be, not just significantly slower, but orders of magnitude slower.

And if you can't fit everything in RAM, the difference between RAID 0, and a single fast disk is going to be relatively tiny.

It's kind of a critical point situation. Unless you get to a certain point, anything else you do doesn't matter. Also Windows NT variants can dedicate 75% of system RAM (3GB on 32-bit systems) to a program using a registry switch.

I would recommend, as others have, looking for boards that support more RAM. Most contemporary boards for Intel CPUs offer 4 DIMM slots. You could also look at single or dual Opteron boards. I would sacrifice CPU power and everything else to get enough RAM in there. If you're swapping all the time, you' CPU may not matter that much in terms of productivity.

A dual CPU Opteron board with two 240s could handle 4GB in 512MB sticks, and would not be inordinately expensive. You could actually save money by going this two CPU route over other 1 CPU routes since 1GB dimms are so expensive.

I recommend this because:

It's much, much better to be CPU limited than disk limited. If you're disk limited in Photoshop and you're swapping parts of the image in and out of memory, the performance of your workstation, no matter what is in it (could be quad Opterons), could likely be qualified as 'unresponsive' most the time. I have worked with PS under such circumstances before and it's unpleasant. It happens on my system if I try and do anything other than work in PS while it's open.

Finally, I have run PS 7 under Linux in Wine and it works quite well. I wouldn't recommend this route for a corporate workstation, but Codeweavers supports PS 7 (read tech support) in some of their products and may be worth consideration. A division of disney runs PS on linux workstations. I figured I'd toss that out there. The AMD64 linux distros aren't great at the moment, but they're farther along than Microsoft's Windows appears to be.

Finally, and most importantly,

could you give us the specs on your current workstation and describe the situations in which performance is suffering. Considering what you are planning on upgrading to, I can't imagine your current workstation is handling this task very well at the moment.

LOST6200,

As ylu knows, the scratch disk is usualy teh limiting speed factor

I am surprised that you say this because, in my usage, I have never found this. The scratchdisk is only limiting when undoing things, otherwise PS commits changes which are dealt with in the background by Windows, causing no perceptable performance hit on any operation in PS.

Of course my usage may be entirely different than yours, for what its worth:

I use PS 7.

With files that start as 70MB *.tifs, but quickly balloon two many times that. They are rarely larger then 200-250 MB. I have 512MB of RAM.

Usually the work is clearly CPU limited and the queue depth rarely goes above 1.

Do well.

Jonathan Guilbault.

Share this post


Link to post
Share on other sites

I can see that you are extremely limited in the budget they have alloted you. I'm kind of shocked they would not consider investing a little cash in a workstation doing this kind of work. But I guess times have changed eh.

You can definitely get Xeon boards with 6 or more DIMM slots too. You would only need one CPU in that case, which could save you some money.

The problem is, most motherboards that support the kind of RAM you need cost as much as everything you've specced out so far :o .

LOST6200 has already pointed out the ideal situation in my mind, which would be for you to get a Mac. You can cram it with cheaper 3rd party RAM if you do careful research and desperately need to keep costs down.

Really the budget is not suitable to the task. That's money for a nice office PC, not a workstation for the kind of work they want you to do. There are lots of deals in computer performance these days, but sometimes you just have to lay down the cash.

P.S. What have you been getting by with, and for how long? Your experience with the bottlenecks in your workload would be invaluable, especially considering the limitations of the budget.

For example, what slows you down the most?

1. Applying filters/effects/layers.

2. Undoing/Redoing

3. Simply browsing the image.

Do well.

Jonathan Guilbault.

Share this post


Link to post
Share on other sites

Oh ya, I wouldn't overclock anything either. It's a corporate machine... I don't think they'd be happy with you spending their money that way. Besides, it's not like you're going to be CPU limited very often.

Share this post


Link to post
Share on other sites

---> LOST6200: :o No, no...! Mac bad, mac pff... :-) I joke but as my dtp experience shows - good configured pc is far more universal for me... i'm little discouraged with macs. And ofcourse buying one isn't possible right now($, $, $ - not only direct expenditures on hardware, also software). Nevermind - Mac is out of possibility. I can't aim to 64bit right now too... 'cause AMD64 processors are still quite expensive. So decent AXP 2500+ is the best i can get now.

----> Big Buck Hunter: Yes, i'm going to do this as soon as i get my hard-stuff in set-it-by-yourself state. I'll tell you guys about my scores/conclusions after first few hours of tests...

-----> Gilbo: Thanks for such long answers :) This is really helpful, i know that i have landed in the proper place. I think... i have to explain why i'm so excited about working on such "future" configuration as i listed above. My present ""workstation"" on which i made many landscapes (starting since DX7 times, until now... when we work with Renderware engine (see GTA3 for example)

- Intel 815 chipset based Gigabyte mobo (office standard)

- Pentium III 866@950MHz

- 512MB PC-133

- GeForce 2MX 64MB

- 2x 5400rpm drives (U6 and one of older WD models)

Sounds prehistoric a little? ;P Yah.. i know. It's a shameful to say but my hardware was not upgraded for more than 3 years. The company i work for started with really-really low budget games and other appz so there was no need in upgrading hardware - "we make budget games = we work on budget machines" (management aproach to this subject..) But times changes... we gain experience.. and if The Company want hit higher market, our old hardware need to be.. hmm... completely changed.

Summaring... Using such slow machine with completely unproportional piece of soft-meat... learned me a lot about "how spend creatively about 20 minutes when my file loads or writes..."

The main delays when i work with a main landscape file (up to 2GB PSD) are:

1) load, write times. It's really too long... In addition during this process - my system freezes, i can't really do much more then.

2) browsing, editing the 6000x6000pix image. It's really frustrating - drives clangs, ratlles, whirls.. screen hangs, refreshes in dozen of seconds... my cpu surely can't take a breath too..

That's all i think... Other activity - adding layers, filters is not such problem... as above things.

Gilbo... dual Opterons or Xeons are out of range for my company right now. Belive me.. that i will be very happy with AXP2500, 1,5GB RAM, R9700 and dual small-Raptors. It will be great upgrade for me.

After posting here.. i contacted our local store.. in order to ask for 1GB modules availability. I get answer that is no problem - Kingston KVR333X64C25/1G is in stock. So maybe i get one of these for good start... and add two more dimms in few months, maxing my RAM to 3GB...

Another question has born: What drives? Two smaller Raptors or two/three bigger 7K drives (Maxtor +9 for example)? I'm really interesed in these Raptors... benching them striped and in solo mode, benching in my real work-condition (saving, loading PSD files, editing with various scratch configurations).

Upff... i'm going to sleep now. Enough for today. Have a nice day folks!

agnus/ Poland

PS. Sorry for poor english, i hope i was intelligible enough... :unsure:

Share this post


Link to post
Share on other sites

Take the two drives out of your old machine and get more RAM!

A faster storage subsystem is not going to get you where you want to be.

Share this post


Link to post
Share on other sites
Take the two drives out of your old machine and get more RAM!

A faster storage subsystem is not going to get you where you want to be.

Agreed. If you could find a way to get more RAM into your current system and not upgrade anything else, I'd recommend it.

Share this post


Link to post
Share on other sites

Are you joking guys? I'm going to change all this i815 stuff right away! I think it will happen soon, maybe in 1-2 weeks.

Besides.. i815 Gigabyte mobo has a limitation of 512MB RAM.. I reached max here.

I'll try to install 1GB modules in this new A7N8X-E-Deluxe mobo... so i should be able to reach 3GB as soon as possible..

Share this post


Link to post
Share on other sites
Also Windows NT variants can dedicate 75% of system RAM (3GB on 32-bit systems) to a program using a registry switch.

...

Finally, I have run PS 7 under Linux in Wine and it works quite well.  I wouldn't recommend this route for a corporate workstation, but Codeweavers supports PS 7 (read tech support) in some of their products and may be worth consideration.  A division of disney runs PS on linux workstations.  I figured I'd toss that out there.  The AMD64 linux distros aren't great at the moment, but they're farther along than Microsoft's Windows appears to be.

FYI, the app must be compiled with support for 3GB to recognize more than 2GB of RAM. I don't think Photoshop supports this, but you can check. This would still apply to an instance of Win32 PS running in WINE.

Share this post


Link to post
Share on other sites

I just thought now...

That 3x Seagate 7200.7 80GB disks cost is comparable with 2x WD Raptors 36GB... Maybe for my work is better to have three physical disks and much more ram? I decided to start from 1GB module and add another two in near future so memory would not be a problem.. And with three Seagate's i could be able to spread all page, scratch files across these drives? And i could have much more space available with 3x80GB (240GB) Seagate's tha with 2x36 (72GB) Raptors..

--

agnus

Share this post


Link to post
Share on other sites
I just thought now...

That 3x Seagate 7200.7 80GB disks cost is comparable with 2x WD Raptors 36GB... Maybe for my work is better to have three physical disks and much more ram? I decided to start from 1GB module and add another two in near future so memory would not be a problem.. And with three Seagate's i could be able to spread all page, scratch files across these drives? And i could have much more space available with 3x80GB (240GB) Seagate's tha with 2x36 (72GB) Raptors..

--

agnus

Sounds ok to me. Honestly, until you get the RAM situation taken care of, anything else doesn't make much of a difference.

The 2 things you said are really slow are affected most by RAM, the things you said are "ok" are the things that your processor is doing.

Share this post


Link to post
Share on other sites

Oh, what is it that you need the 9700 card for? Seems a bit overkill for a workstation. Granted, the MX card would suck at both 2d and 3d, but find a good Matrox card or low-range 9200 instead, and use the saved money on a better platform maybe?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now