agnus

Photoshop 2gb+ Files, Need Advice

Recommended Posts

---> rp: It's not so easy... I make landscape then throw it to the 3D engine.. and use 3D editor to sculpt whole terrain. With GF2MX FPS drops down to 2-5p very often.. which is not acceptable. I need fast GPU, with decent amount of VRAM (128) to load textures and smoothly animate that geometric circus (terrain, objects, particles..). The second important thing is quality of what i see in high resolutions. I use LG Flatron 915FT Plus... i'm stuck with 1152x864pix now... i hope with this Radeon i would be able to hit at least 1280x1024pix.

Share this post


Link to post
Share on other sites

LidlessEye,

FYI, the app must be compiled with support for 3GB to recognize more than 2GB of RAM. I don't think Photoshop supports this, but you can check.

You are absolutely correct. I totally forgot that...

Photoshop must be configured to use a certain percentage of system memory. I do not know if you set it to 75% and you have 4GB if it will go above 2GB. I would check, but I don't have 4GB of RAM... let alone a motherboard to put it in if I did.

Agnus,

My RAM recommendation still stands --it is worth getting the RAM at any cost to the rest of your system... but if not:

I suggest you get several, cheap 7200RPM drives. To my knowledge the swap file will contain the image data that doesn't fit in RAM and the scratchdisk will contain undo information, individual, seperate layers, etc. Swapfile performance is more important for most of your bottlenecks. I'm sure someone will clarify my interpretation of the relative uses of these swapfiles if I have made any mistakes.

Current Maxtor drives have the best price to STR ratio. See the performance database. I would recommend several smaller capacity ones. These will offer more performance for your usage than spending the premium for Raptors. You'll get more STR this way, per dollar than with Raptors... way more. If you can afford it, using low capacity 7K250s would be ideal. They have significantly better seek times (bear in mind this is not a primary consideration for your usage) and appear to have superior firmware algorithms by SR's tests.

Now. The outer partition of the drives should be reserved for the swapfile. This partition should not be striped. Windows will distribute the page file across these drives if you assign the page file to all those partitions on its own and use the disks more intelligently than a RAID 0 would allow it to. These partitions should be as small as possible with consideration that ther must hold the upper limit you set on swap size.

The scratchdisk should be on the next partition of those drives. These partitions should be striped. Yes, I said it... I bet a lot of people didn't see that one coming.

Finally, if loading and saving is a big deal, stripe the final third partitions of the drive and use it as a working space for currently used files. Set up a back up utility (Windows Backup should be fine) to periodically copy the contents of this drive to a real storage drive (you'll need another drive) or to space on your company's server --just make sure there are copies off the striped set.

That's about it, I think. Bear in mind this is only an attempt to do as much as possible with too little. It is not in any way a very satisfactory solution, but I think it may be the best one you can afford.

Remember, if you can, buy RAM. The more the better. Even though PS will only use 2GB, Windows can use the rest as file system cache. This will still help you a great deal.

Do well.

Jonathan Guilbault.

Share this post


Link to post
Share on other sites
---> rp: It's not so easy... I make landscape then throw it to the 3D engine.. and use 3D editor to sculpt whole terrain. With GF2MX FPS drops down to 2-5p very often.. which is not acceptable. I need fast GPU, with decent amount of VRAM (128) to load  textures and smoothly animate that geometric circus (terrain, objects, particles..). The second important thing is quality of what i see in high resolutions. I use LG Flatron 915FT Plus... i'm stuck with 1152x864pix now... i hope with this Radeon i would be able to hit at least 1280x1024pix.

Wouldn't you be better off getting a Wildcat, FireGL or Quadro type card? Even Wildcat III's aren't that expensive.

But it must suck, be on such a tight budget. I'm suprised that the management doesn't understand that time = money. If you can get a better wkstn, you can save time doing what you do, which equals more productivity and more money?

Share this post


Link to post
Share on other sites

Chewy509 said,

If you can get a better wkstn, you can save time doing what you do, which equals more productivity and more money?

I agree entirely Chewy.

It's slightly ridiculous considering how little money it would cost, relative to the amount you're going to pay the employee. Increase the budget a measly $500-1000 and the difference would be tremendous. Just enough to buy two slow Opterons and fill a board with 512MB dimms. The cost of the workstation is tiny relative to hiring someone to work on it.

Share this post


Link to post
Share on other sites

---> LE, Gilbo: It's correct that PS cannot use more than 2GB of memory, as FAQ on Adobe.com website speaks:

"Photoshop supports a maximum file size of 2 GB and a maximum dimension of

300,000 x 300,000 pixels per image. In addition, Photoshop can access 2 GB

of installed physical RAM."

and..

"Photoshop CS allows for images with a maximum file size of 4 GB for TIFF

files, and almost unlimited file sizes for images saved in the new native

Large Document Format (.psb). Photoshop CS can only access 2 GB of RAM, so

large files will need to use considerable scratch disk space. Large Document

Format files cannot be read by Photoshop 7.0.x or earlier. ."

So all is clear now. I will need to max my RAM to 3GB - and confugure my PS to use this 2GB and rest will be left for Win2K to consume..

---> Chewy, Gilbo: I understand that such professional graphic card would be the best for me... but as i said before - i'm very limited in money (my company is - to be correct). I cannot aim for video card for more than 250$max. So i decided to choose very flexible, quite fast and with good image quality card - Ati Radeon 9700 128MB. Matrox isn't for me because of simple reason: It's too slow in 3D. Other "pro" cards are to expensive. So my only choice is to obtain decent, "consumer market" card and to be happy with smooth FPS both in 3D Editor (my work) and newer games (eghm... "reaserch activity" ;P ;P )

--->Gilbo: Thanks for advise about drives. Three nice 7200 drives would be better than two expensive Raptors. I'll follow your instructions when it will come to partitioning.

--

best regards,

agnus

Share this post


Link to post
Share on other sites

Wow, although I feel for you (having to do that work on that machine - check my sig). It sounds like you have a good feel for what you need to do. I cannot disagree with any of the advice in this thread...I love this board ;)

Share this post


Link to post
Share on other sites

I've did a bit of work on the Blue Marble NASA textures (huge raw image files), and I must concur with those espousing RAM above all else. Swapping around a couple gigabytes just isn't a very productive experience.

Share this post


Link to post
Share on other sites

Aqnus, keep in mind also that the nForce2 chipset only supports 16 chips per DIMM, if you are using stacked modules (which most of the 1 GB Modules are) you may end up with not seeing the upper 512 MB on each module.

This is where the nForce2 chipset is not comparable to most other chipsets but bottom line is that if you try to use 1GB modules, you need to make sure that the modules are not using stack technology or if you get those, make sure that you can return them unconditionally.

There are a number of different stack technologies out there, with some of them being able to overcome the aforementioned chipset limitations but it is really a case by case basis whether it works or not. Personally, I have not been able to get 1 GB modules to work on any of the A7N8X boards that I have had (about 1/2 dozen different revisions and, I think 8 different 1GB module designs). I don't want to discourage you, only try to save you some headaches (since these issues are documented in the nVida data sheets as well)

Share this post


Link to post
Share on other sites

----> khuzdand: Yaa.. i know it already ;) I have to edit layered maps which are 6000x6000 in size... My two, poor, old 5400 drives are doig their best.. but i have really a lot of spare time at work drawing funny figures with paper and pencil while my landscape writes or loads ;)

-----> unregistered: Oh my.. :/ I got headache now :( I will be carefull obtaining these big modules now. This one i have in range, as proposition from my local store is called "KVR333X64C25/1G". I downloaded a data sheet for this module from here: http://www.valueram.com/datasheets/KVR333X64C25_1G.pdf and i'm confused.. will work or not? :(

Okee... let start my day at work... i have to return to my "workstation" and build some beautifull scapes for you guys (for games ;)

Cya,

agnus

Share this post


Link to post
Share on other sites

In the first paragraph of that PDF document, it says:

The components on this module include sixteen

64M x 8-bit (16M x 8-bit x 4 Bank) DDR333 SDRAM in TSOP packages.

So yes, these should work fine ;)

Share this post


Link to post
Share on other sites

Oh yes... i seen that but there was to many numbers :) Thanks for confirmation!

Share this post


Link to post
Share on other sites

Agnus, trust me when I speak about Photoshop, look at www.Photoshoptechniques.com for a known name (and feel free to ask a similar question over there too)... and I visit adobeforums.com everyday (as Pierre Courtejoie, plus several other PS fora ;)

In your statement about Photoshop from the FAQ, be careful: Photoshop 7 and earlier are limited at 30.000 pixels in every dimension (and up to 24 alpha channels) PS CS can go to 300K x 300K pixels.

LOST6200, sorry, but you are lost about Photoshop on a mac :D ! Photoshop on every platform is only able to use 2Gb of ram... of course, the rest could be used for other apps, or even a scratch disk, but it is not different than using a Xeon with a lot of ram slots, or a dual AMD 64 board with 8 ram slots...

The Adobe software engineers stated that they need to have a stable operating system before they can test a 64 bit version of Photoshop. this is valid for both platforms. Remember that MACosX is not entierely 64 bit!

Share this post


Link to post
Share on other sites

If you really want to use Raid for the scratch file, why only consider the raptors?

you can get a STR close to saturate the ATA bus with most modern hard drives now. Also, I recommend to test as well the use of 2 hard drives not raided for the pagefile.

Thus, you could get more disks for the same price (one for apps+OS+pagefile+a data partition and maybe another one for more data, and security...).

Remember that Adobe recommends to save the data on the local HD then save it on the network. (And it might be better to copy the file from the network to the local HD prior to open it)

Share this post


Link to post
Share on other sites

Despite my strong support for RAID-0 I believe that RAID-0 is not appropriate for this purpose. Like others have said use multiple drives and spread the work load. The biggest performance gain will be obtained by making sure your drives do not spend much time seeking. Sepparate drives for swap file and scratch disk is essential, prefferably have a third sepparate drive for the OS. My recommendation is 3 80G drives with no RAID-0.

Also, keep the scratch disk unfragmented!!!!!!!!!!!!!!!

Share this post


Link to post
Share on other sites
Agnus, trust me when I speak about Photoshop, look at www.Photoshoptechniques.com for a known name (and feel free to ask a similar question over there too)... and I visit adobeforums.com everyday (as Pierre Courtejoie, plus several other PS fora ;)

In your statement about Photoshop from the FAQ, be careful: Photoshop 7 and earlier are limited at 30.000 pixels in every dimension (and up to 24 alpha channels) PS CS can go to 300K x 300K pixels.

LOST6200, sorry, but you are lost about Photoshop on a mac :D ! Photoshop on every platform is only able to use 2Gb of ram... of course, the rest could be used for other apps, or even a scratch disk, but it is not different than using a Xeon with a lot of ram slots, or a dual AMD 64 board with 8 ram slots...

The Adobe software engineers stated that they need to have a stable operating system before they can test a 64 bit version of Photoshop. this is valid for both platforms. Remember that MACosX is not entierely 64 bit!

If photoshop can't use 3gb with the /3gb or 4gb under wow it has a few simple bugs that need to be fixed.

I'll check out the tryout version tommorrow, if I can talk myself into playing with the memory in the quad opteron systems.

Share this post


Link to post
Share on other sites

Meh, you need to use an athlon 64, and:

1) install 8gig ram

2) setup a 5.5 gig RAM drive

3) make photoshop use the RAM drive as the Primary scratch drive.

:P

Share this post


Link to post
Share on other sites

sPECTre,

you can get a STR close to saturate the ATA bus with most modern hard drives now. Also, I recommend to test as well the use of 2 hard drives not raided for the pagefile.

Good recommendations... however, Agnus and I already talked about both those points in those threads and reached conclusions. You were too slow. :P

Patto,

Despite my strong support for RAID-0 I believe that RAID-0 is not appropriate for this purpose. Like others have said use multiple drives and spread the work load. The biggest performance gain will be obtained by making sure your drives do not spend much time seeking.

Obviously you know that I advocate considering such a configuration above all others normally. However, if you read my post on how he should partition those you will realize that seeking will be limited. Additionally, the swapfile and scratchdisk should not be hit at the same time very often. They are used for different purposes. Considering his limited budget I believe he will see better results in the configuration I advised earlier.

Yes, I am recommending striping without parity... :o

Do well guys.

Jonathan Guilbault.

Share this post


Link to post
Share on other sites

I have done some more research.

I believe you do not need to stripe the scratchdisk partition. Just set it up like I advised for the swapfile --tell Photoshop to use multiple disks. I checked on my version and it allows you to set primary, secondary, etc. Limit the sizes of the scratch disks so all the disks get used on your biggest images.

I don't really know if this configuration will perform better for you than the one I advised in my earlier post. If you feel like it, you should test them. I would simply use dynamic disks for striping the partitions (by the way, if you didn't know that before).

Really, there should be little difference between these two options.

Finally, the swapfile and scratchdisk partitions on these disks should be the first partitions on the disk (one after another). Because the boot partition has to be the first partition on a disk you will need an additional drive for your system drive. I suggest using a drive from your old system since the performance of this drive will not be critical. This will allow you to invest the money you have in drives whose performance will be critical. Additionally, assign the Windows swapfile to this boot disk as well, and use it for your applications. Frequently people advise seperating the system/programs disk from the swapfile. This advice is not well thought out since once the OS and programs are loaded that disk is pretty much idle. You may as well swap to it.

Goodluck and do well.

Jonathan Guilbault.

Share this post


Link to post
Share on other sites

Actually, those drives are 5400RPM drives. Don't put a piece of the swapfile on it. While Windows understands to use the least busy drive when swapping to multiple drives, it doesn't know to prefer faster drives. The advice at the end of my last post may actually slow your system down.

Right on. I think that about does it.

Share this post


Link to post
Share on other sites

Actually, I'm not done. :(

I always thought that PS only used the scratchdisk for certain particular things (like undo history etc). However, I have determined using tests on my system that it uses it for everything that won't fit in RAM. The swapfile sees no action from PS at all...

Therefore, the optimum set up would be to put the scratchdisk files on the outermost partitions and the Windows swapfiles on the next innermost one. The swapfile will likely not be used at all, unless you load other applications while using PS --which I would not advise. This is a good thing overall, and intelligent work by the Adobe designers.

I think that's the best I can do given your budget.

Share this post


Link to post
Share on other sites

Oh, god. I'm sorry. I meant to post this earlier but forgot while I was playing with PS on my system and watching what it did.

I noticed you said you're getting 3 disks. What sizes? Get the most spindles you can. Use 5 smaller disks instead of 3 larger ones if its an option.

There are many cheap RAID controllers that will give you lots of channels if you need extra ones. Use dynamic disks to stripe the partitions even if you get a little RAID controller --you don't want to stripe the whole drives. Just the scratchdisk partition (maybe, I'd try it without as well --performance without depends a lot on the intelligence of PS's swapping decisions) and if you feel like it your temporary file storage partition.

Share this post


Link to post
Share on other sites
Actually, I'm not done.  :(

I always thought that PS only used the scratchdisk for certain particular things (like undo history etc).  However, I have determined using tests on my system that it uses it for everything that won't fit in RAM.

Now you've got the idea! Sorry if I did not explain it befroe, but there was pain and hurrying. You can also understand why I like having 4GB of RAM and using half for the RAMDiks scratchdisk. ;)

Angus, I am sorry for your plight. In any occupation it is difficult having to work without the proper tools.

B

Share this post


Link to post
Share on other sites

Photoshop, in fact uses the the page file (Windows XP does not swap) as a backing store for the physical RAM. Presumably, if you're going to be maxing out memory with Photoshop, there will be some page file use related to that.

Share this post


Link to post
Share on other sites

You should demand more money for this workstation. It's just ridiculous for you to be on such a limited budget when your needs are so demanding. Heck, even the best machine isn't really enough for your needs! Buying a more powerful machine would be totally cost effective because it would make you so much more productive. Print out this message and give it to your procurement guy or boss so that he will know what an idiot he is.

Share this post


Link to post
Share on other sites
You should demand more money for this workstation.  It's just ridiculous for you to be on such a limited budget when your needs are so demanding.  Heck, even the best machine isn't really enough for your needs!  Buying a more powerful machine would be totally cost effective because it would make you so much more productive.  Print out this message and give it to your procurement guy or boss so that he will know what an idiot he is.

Absolutely. Explain that by investing more in your worstation, you can finish your work twice as fast, so he can cut you back to half-time....oh, wait...

...nevermind ;)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now