qawsedrftgzxcvb

Hard Drive Bottleneck In Game

Recommended Posts

I'm sorry to say that I disagree with the two previous posts entirely.  The best coding in the world cannot fit arbitrarily huge maps into a limited memory.  It is ALWAYS possible to make a map and it's textures larger than the available memory, and no amount of coding can reduce that (save for incredibly efficient compression of textures and maps, but they already do that to some degree). 

When game designers built games that would fit into system memory, reviewers complained that the levels were too simple.  Now they make them much more involved, with much larger maps, and fewer of them.  Each level is now much more challenging and involving, BUT you do have to page map data...

What has happened is the average storage for a decent gaming system is now so much faster that designers figured it was up to the task of a quick load in the middle of the game  - something they couldn't assume even a coupld of years ago.  To take them to task for it, when they are probably responding to player input for "larger levels" is pretty silly.

Future Shock

I've made maps for halflife and quake.. I've also programmed JAVA, C++, BASIC, and scripts.

There are techniques to load maps in the background that should not produce stuttering.

Additionally, maps aren't usually all that big in terms of MB size. The main problem is complexity, It would be possible to make a quake1 map that would be too complex for any current graphics card.

That WOULD be the programmers fault. The programmer of the map is responsible for making sure it will run smoothly on the machines the game was designed to be played on (or the least common denominator still playing the game).

If your programmers are making maps too large or complex for the systems around then they need to optimize the maps. My guess is that the maps of current games WILL fit into RAM, with a few exceptions (like the single large map of MS train simulator).

For these types of games, the system can get the information ready and try to load it in advance so that when the information is needed, it will be available without the game pausing to wait for the information to load.

Share this post


Link to post
Share on other sites

You don't know the disk is the bottleneck, it could also be processing data it reads. TaskMon would show a CPU spike.

You could also run PerfMon and see what the disk I/O rates are.

Share this post


Link to post
Share on other sites

I doubt the 15.3K disk is the bottleneck. Maybe the controller if it is in a 32/33 slot. If the game cannot play on that drive, ATA would never handle it. Do you disable your Antivirus software during playing. I have seen it pause a game.

Share this post


Link to post
Share on other sites
I've made maps for halflife and quake.. I've also programmed JAVA, C++, BASIC, and scripts.

There are techniques to load maps in the background that should not produce stuttering.

Additionally, maps aren't usually all that big in terms of MB size. The main problem is complexity, It would be possible to make a quake1 map that would be too complex for any current graphics card. 

That WOULD be the programmers fault. The programmer of the map is responsible for making sure it will run smoothly on the machines the game was designed to be played on (or the least common denominator still playing the game).

If your programmers are making maps too large or complex for the systems around then they need to optimize the maps.  My guess is that the maps of current games WILL fit into RAM, with a few exceptions (like the single large map of MS train simulator).

For these types of games, the system can get the information ready and try to load it in advance so that when the information is needed, it will be available without the game pausing to wait for the information to load.

If you believe that, then more precisely it is NOT a programming fault, it is a level design fault. Most game companies (pro ones at least) have different staffs for both. Because it all falls under the rubric of playability, it is also a program manager's fault too, as he has overall responsability for both sides of the house.

I would be quick to point out that both HL and Quake were games that were put out before the majority of gamers had fast disks, and were designed for that assumption. My basic tenent is that the assumptions are now changing, and that fast 7200 or 10000 RPM drives, often in RAID 0 configs (duh!), are now de rigour for high end gamers. Current games are targeting that as hardware, simply because the people that buy the most games (high-value consumers, in marketing speak) tend to have the best rigs. So do most game reviewers.

And as for loading maps in the background, that obviously only works if you have enough memory to store them both in the first place. Since the whole point is that they DON'T HAVE ENOUGH MEMORY, well...

Future Shock

Share this post


Link to post
Share on other sites
btw, that's not a bright thing to say.. the problem is obviously poor coding on the software developers end in terms of HALO for PC..

I doubt the average machine has 4GB of RAM to hold the entire game. They aren't the best coders (oooh believe me... their best code are copied from fans) but they have some intelligent coding monkeys.

Try a NO-CD patch

I've been looking for one for sometime. Links are all broken so far.

Add 1 GB of RAM and see how it plays. You could get a 64/66 or PCI-X 64/133 slotted motherboard and a controller to increase throughput.

RAM already at 1GB. Adaptec 29160LP already in a 64/44 PCI slot ;) . I wouldn't do it any other way.

You don't know the disk is the bottleneck, it could also be processing data it reads. TaskMon would show a CPU spike.

Neither one of my CPUs hit 100%. Both CPUs barely reach above 50%. CPU 1 usually is less than 90% and the 2nd is less than 20%. Fortunately the game is not SMP aware. The last time this company programmed SMP, they created some of the worst problems a program could have.

Do you disable your Antivirus software during playing. I have seen it pause a game.

Only ATI Omega driver software runs in the background. I disabled that, too.

You could also run PerfMon and see what the disk I/O rates are.

Good idea....

TaskMon says page file is at 322MB.

The results are intriguing to say the least.

-The nice little game has pages at nearly 100pages/sec around 30% of the time. I turned off the page file and I still get this. Even with 700MB of RAM free this happens. :huh:

-Pages hit 100 when I move around the map at anytime and never go down under 10% when game is active.

-Physical disk ques when loading map jumps to an average of 10 ques/sec.

-When I walk around transfer and reads jump up to 100. Transfter stays at 100% most of the time.

-Disk I/O stays pretty low.

I'm starting to think the problem might be the paging all the time.

I'll try a no CD hack when I find one and see if that helps. I know it helped decrease load times in last expansion, but last expansion wasn't this bad.

Share this post


Link to post
Share on other sites

I placed my 4GB paging file on an old 18GB 15K U160 drive along with the Disaster recovery images, Win XP Pro files, and temp files. It helped with paging being on their own hard drive.

Share this post


Link to post
Share on other sites

I've noticed the same behavior with NWN since it came out.

I cannot say if the game is checking for the CD, but I can say I play NWN with my CD drive off (external case) -- after loading of course. ;)

I generally use CD images but every time a new game comes out it's time to upgrade the CD software...so I'm getting tired of it.

Almost all games I play now drop my disks to there collective knees for a second. Pretty amazing.

I used to play NWN on a dual PIII w/ 2GB of RAM, and now a 3.2GHz P4 (although it says 2.8 on the chip. :D). It has always taken about 1 second to load when new creatures / effects come into view. That huge upgrade and it acts the same. :)

DogEared

8^)

Share this post


Link to post
Share on other sites

A somewhat different approach (if you have the hard disc space) is to decompress the game files (maps especially). This means that the decompression step is not needed when the maps are loaded. This can make an amazing difference for some games provided such extraction utilities exist.

Share this post


Link to post
Share on other sites

MaZeR, there is no such thing... yet to my knowledge.

DogEared which NWN are you talking about? Neverwinter Nights and Shadows of Undertine took no time at all. 1GB of RAM, and a 15K drive took it no prob. Hordes of the Underdark keeps paging. I have no idea why. I am trying to disable the pagefile to hopefully stop the slowness.

This is not a CPU bottleneck. A dual 2400+ should have no problems. I tried the game on a machine with 4GB of RAM and a dual 2Ghz, S2885 mobo, Opteron server . The sucker connected to a fibre-channel SAN and it still had the same slowdown.

If anyone has any ideas how to disable a page file lemme know. I tried disabling though windows XP's tool but it didn't work.

Share this post


Link to post
Share on other sites

If they are memory mapping the files (quite possible), then the paging has nothing to do with the page file and more memory should help. I didn't say fix.

You can also set the pagefile to a small fixed size. Then once it fill, it shouldn't be use much. It might help, it might not.

Try a tool that lets you run off the cd image.

Are you sure that it isn't loading the files from the cd?

Share this post


Link to post
Share on other sites

me gonna try this:

System Key: [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\
Memory Management]
Value Name: DisablePagingExecutive
Data Type: REG_DWORD (DWORD Value)
Value Data: (0 = default, 1 = disable system paging)

hopefully it will work

Share this post


Link to post
Share on other sites
YARRGH!!

I did what I said, rebooted and looked at the task manager.

I see 96.2MB for a pagefile!!!

Is there no way to disable the accursed pagefile.

What you told the kernel to do, was not to page itself (the OS) out to disk. It will still page out to disk, your applications and data if it sees that memory is getting low.

Share this post


Link to post
Share on other sites

IIRC, that reg key is for older version of windows.

Windows page files are not that stupid. It will not load data from a disk and then page it out unless it was modified. If anything it will reload the data from the disk and never put it in the page file. It doesn't page data without memory presure or without the process being used.

You have neither problem. This is most likely a buggy game.

Try Daemon Tools

Share this post


Link to post
Share on other sites

I have play the first two installments, nothing whatsoever happened interms of serious swappings. But further to my point, it takes a very little air-headed programming mistake to topple the balance of powers ;) Just to name an example, it makes a hell lot of difference to a mis-programme ;) if you change the AGP Aperture size alone. :unsure:

Share this post


Link to post
Share on other sites
YARRGH!!

I did what I said, rebooted and looked at the task manager.

I see 96.2MB for a pagefile!!!

Is there no way to disable the accursed pagefile.

What you told the kernel to do, was not to page itself (the OS) out to disk. It will still page out to disk, your applications and data if it sees that memory is getting low.

RC on my com, props, perfor..... you know the rest.

For XP and NWN, make sure that you have at least 1GB before you disable the page file.

Thank you for your time,

Frank Russo

PS, Ken was dead on about that reg key. Works in NT4.

Share this post


Link to post
Share on other sites

*qawsedrftgzxcvb kisses FluffyChicken and promises to not eat another chicked for 2 months *

THANK YEE!!

Load times are fast as hell again. The only bad load times as the large custom ones I create.

Share this post


Link to post
Share on other sites

to not eat another chicken* for 2 months

Hum NWN:HotU (Home of the Underdogs )

When I called up most stores they thought I was saying "Whores of the Underdog".

"Is that a porno sir, because we don't sell them here."

Share this post


Link to post
Share on other sites

HA I had similar experience. I ended up buying at a game store anyway but it was fun listening to people repeat what I wanted!

"Whores of Underdog?" Oh, Sweet Polly! LOL!!

My personal favorite:

"Was that Hordes in the Dark or Whores in the Dark?"

Looks like the No_CD fix saves the day again. When will publishers learn copy / pirate protections are just a major pain to game owners and DO NOT stop piracy?? *sigh*

DogEared

8^(

Share this post


Link to post
Share on other sites

it was certainly not the reg key. And the problem was certainly not HDD bandwidth or a "technological limit". The problem was poor coding/decision making on the game makers end.

Share this post


Link to post
Share on other sites
it was certainly not the reg key. And the problem was certainly not HDD bandwidth or a "technological limit". The problem was poor coding/decision making on the game makers end.

I think I expressed myself incorrectly.

First off no it wasn't the reg key. Secondly it is not feasable to code a game for the highest end hardware. If they coded the game for a RAM drive and a system with a quad configuration, they could. I just wouldn't be feasable.

It is a technological bottleneck in the sense they have a limited RAM to work with. That's why there must be disk access all the time. Plus the game seems to check the CD quite a few times to make sure you didn't take the CD out.

Share this post


Link to post
Share on other sites

Just reinstall my NWN, it got some horrible BSOD should one try to load an expansion pack character saved game in the original NWN game without the expansion being installed. That could possibly indicate something very wrong has happened in its coding, since this is one of the very few BSOD I saw in XP.

BTW, probably for N-players, ;) , my level 20 Aluvian Darkstar character,

Alu Darkstar

Share this post


Link to post
Share on other sites

I'll second the "W00t" for the "NO_CD hack."

My stuttering / hiccups disappeared completely. That makes me happy & sad.

I'm happy that the game just flies now, but I'm sad that the code exists in the first place.

It's stupid to check for the CD when the game apparently doesn't care if the CD is there or not. As I stated before, my CD Drive is off. There is *NO* way for the game to see the actual CD, aside from it's own caching.

DogEared

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now