[MUD-Dev] Benchmarking MUDs

[Chris Gray]

> From: Matt Mihaly <the_logos at achaea.com>

> How could we benchmark MUDs? In other words, how could we test
> them under the exact same conditions?

That's one of the key questions. Others include:

    Just what aspects of MUDs could or should be benchmarked?
    How do you compare a softcoded MUD to a hardcoded MUD?
    How do you compare a graphical MUD to a straight text MUD?
    How do you abstract out the network connection in use?

I think that for a first attempt, not too much accuracy is
needed. In fact, knowing things to within an order of magnitude
would be a good start.  Doing that shouldn't be too hard, since most
folks have access to systems of roughly the same performance
characteristics. Run the MUD on some fairly standard home system,
e.g. an X86 around 300 Mhz - 1 GHz, with 128Meg - 512Meg RAM and a
reasonably fast disk.

Most MUDs use good chunks of memory, for storing the whole world, or
some cached subset of it. What is the memory footprint of the MUD,
given some number of active clients doing "standard" things? 
(E.g. conversing, fighting, exploring - whatever is the norm for the
MUD.)

In the same situation, what is the rate of disk I/O that the MUD
does?  What is the mix of read/write versus seek (i.e. how efficient
is the I/O, in terms likely to reflect the disk head seeking back
and forth)?

For softcoded MUDs, how efficient is the softcode system? Write some
simple counting loops, and sequences of subroutine calls (just use
recursion if its supported), and see how long they take. I assume
here that no softcode system bothers to try to optimize empty loops
etc., since there doesn't seem to be any reason to do so.

Since MUDs are designed to handle multiple clients, and to send
stuff back and forth over the network, that needs to be looked at
too. This is a harder thing to do, however. A set of standard robot
clients could help here, with a bunch of interested people running
them against the "target" MUD of the day. However, the internet
connection of that target MUD is going to be pretty important
(unless the MUD system is such that its internal operations are slow
enough that network traffic is not its bottleneck). One possibility
here is to design the robot clients to be efficient, and then run
them on the same machine as the MUD server. Then, system measurement
tools can show what proportion of the system's resources are
consumed by the MUD server and what by the robot clients. This can
run into problems, however, since some MUDs can support hundreds or
perhaps thousands of connections, and the test systems are likely to
have trouble with hundreds or thousands of test programs all
connecting to the one server program. Perhaps the robot client
programs can be designed to each simulate some larger number of
clients.

My MUD is softcoded, and so I was able to do a fair amount of stress
testing internally, using robot NPC's. I made them wander around,
fight, pick things up and drop them, etc. This isn't too much work
to set up, depending on the softcode system, but won't work as well
for hardcoded systems, since such NPC's would likely have to be at
least partially hardcoded.

Is anyone besides me actually interested in doing this?

--
Experience should guide us, not rule us.

Chris Gray     cg at ami-cg.GraySage.COM
               http://www.GraySage.COM/cg/
_______________________________________________
MUD-Dev mailing list
MUD-Dev at kanga.nu
https://www.kanga.nu/lists/listinfo/mud-dev