[MUD-Dev] Re: processors

[diablo at best.com]

J C Lawrence wrote:
> diablo <diablo at best.com> wrote:
> 
> 
> You are barking up the wrong tree in the wrong forest, but are on the
> right continent.  Sorry to be so blunt, but I used to do this sort of
> stuff for a living.

Not a problem.

 
>   I've seen too many performance have faster systems thrown at them,
> resulting in ___zero___ performance increase because the real
> performance bottleneck was in disk IO or some such, and that aspect
> didn't change at all in the new system despite the fact that the new
> CPU was 20 times faster.  A really bad example of this BTW was where
> the real problem was the striping size used on the RAID arrays.  Once
> we bumped the stripe size up to around 8Meg, performance went thru the
> roof.

It's definitely not disk access. We barely access the disk at all. Nearly
everything is held in memory. 

<many undoubtedly erudite questions snipped> 

I snipped those questions above because I wouldn't know how to answer
them.

I am guessing the problem is processor-related becuase I was able to solve
most of the problem by reducing the number of calls per second our main
mobile ai routine gets. This fixed the problem we were having, which was
that our tasks (timed events) were taking WAY too long to go off. A 3
second task would take 20 seconds, but this only happened with a heavy
player load. Our tasks are held in a table indexed to some number (some
sort of ticker in linux? I don't know the proper names for anything). That
table is polled every time the game searches for player input (it cycles
through all our player lines constantly), and if a task is found to be
either overdue or ready to go off, it goes off. The problem comes when
those tasks start taking too long. This of course causes a cascading
effect as more and more tasks build up.

Generally, the way the author of our language/engine explained it to us is
that the engine and language are not Object Oriented in order to be
faster, and in order to allow for more proactivity (as opposed to
reactivity). We will try reduce the drain that things like mobile ai put
on the processor, but it's a pretty heavy task, complete with string
manipulation and interpreting a mini-programming language we have inside
the game.

Could someone like yourself, or someone similarly erudite about such
matters speed things up by recoding? Probably, but there's a lot of code,
and probably a _lot_ that needs doing. We'll certainly try to do it, but
we will also be buying the fastest processor we can afford. I know it goes
against the programmers ethic to dismiss a problem by saying "buy more
memory" or "buy a new processor" but I've committed worse sins I suppose
(though my new coder starts foaming at the mouth everytime I
suggest it. I'm a little frightened that upgrading instead of optimizing 
will kill him.)

> Unfortunately this whole area is a bit of an art.  Very very tiny
> changes made to systems can have massive performance returns (or
> penalties).

In that case, consider me the Thomas Kincaide of programming.

 
> I'm going to generically bet that your system is suffering from almost
> all of the following as they are what I usually see:
> 
>   1) IO bandwidth (likely disk, unlikely RAM, possibly network, very
> possibly IPC).  NB Excessive heap fragmentation can massively
> exacerbate this problem.  'sar' and all the other system analysis and
> report tools, especially the live-system ones are your friends.

Like I said above, it really doesn't seem like it, given that slowing down
mobile ai (which doesn't access the disk at all) speeded things up quite a
bit. 

> 
>   2) Busywork -- just bad algorithm implemenation with pathological
> performance characteristics.  Yeah, it happens, even when we try not
> to.  Profilers are your friends.

Oh yeah, this is definitely part of the problem. I still find big lists of
if-then instead of if-then-else statements in my old code. Do profilers
have to be written for specific languages? My code is written in a
language written and maintained by a friend of mine. Not marketed.

 
<Had to snip more things so that I didn't embarass myself admitting I
didn't know what they mean>

Thanks for your help, JC.
We shall try to optimize, but I'll still install a new processor behind my
coder's back. 
--matt