[MUD-Dev] Thread Pools (was: Re: TECH: Distributed Muds)

[Greg Underwood]

At 11:46 AM 5/24/01 -0700, Scion Altera wrote:
> Wednesday, May 23, 2001, 3:44:32 AM, Matt Chatterley wrote:

>> A model which would work (hopefully) does occur to me.

>>   [Socket connections] - 1 thread per player to read data and pass
>>   on

>>   [Input processing layer] - 1 thread which interprets all player
>>   input

>>   [Event Handler] - Accepts events from systems, orders and
>>   initiatves actions, events, and consequences to keep the
>>   In-Character world running.

> Why do you need one thread per player? Why not just have one thread
> that iterates through the sockets and queues up their input and
> spits out their output buffers? As long as this thread doesn't block
> and isn't allowed to run wild (I'm running mine at one pulse per
> 1/8th second at the moment), it seems to run well enough.

There were a few conversations about using Thread Pools a while back.
I don't have time to search the archive, but it would be a good idea
for those seriously interested.  :)

As I recall, the major concerns on the above model were:

  1) max number of threads available per process will force a limit on
  the number of connections, and therefore the number of players, you
  can have.

  2) Short of the max # of threads the system will support, most apps
  will run into problems once you hit about 50 usefull threads
  running.  The exact number will vary with how much work the threads
  are doing, but as Scion said, there is overhead in creating and
  using a thread.  Thread's still require a context switch (albeit a
  smaller context) whenever they activate and sleep.  You can only run
  so many before the context switches start hurting.

  3) Muds don't lend themselves to Threading as well as you might
  think at first glance.  The best problems to solve with threading
  are those where there is little to no shared data between the
  threads, thus reducing the number of contentions (blocked threads
  waiting for another to finish - IE: Serializing your parallel
  model).  Muds are specifically designed to let players interact with
  eachother and the same world data, which causes problems for
  threads.  Granted, there are situations under which two threads can
  manipulate data w/o fear of clobbering eachother, but I'd argue that
  Muds are not well suited for massive threading.

Don't get me wrong... the key words there were "massive threading."
There are definately things that can be spooled off to a seperate
thread for handling.  Just keep in mind that Threads only give you
benefit if they act independantly of eachother most of the time.  If
you find yourself putting a lot of effort into your locking and data
control, you should think about reconsidering your threading model.

> Branching out from my own current model for a moment, and touching
> on something asnellin at san.rr.com mentioned in your quote, what about
> a thread pool?

> Consider the model where you have a list of sockets that need to be
> updated x times every second. Here is a system I am considering:

>   o Start off with one thread doing the updating, and keep track of
>   the time taken by that thread to do one iteration of the list.

>   o If the time taken exceeds a limit, such as 1/8th of a second,
>   spawn a new thread and divide the list equally between the
>   original thread and the new thread. Ideally, this will cause the
>   execution time for each of the two threads to be less than the
>   limit.

>   o If the time taken drops below a limit on two threads, have one
>   thread die and give its list over to the other one to consolidate
>   the pool.

> And for the sake of sanity:

>   o Kill all threads who have blank lists.

>   o Impose a hard limit on the number of threads in the pool.

>   o Throttle the threads so they don't spin up too fast and eat too
>   much CPU, even if there's only one of them :)

The only problem I see is the cost of creating and killing threads all
the time.  Why not just pre-allocate the pool at startup, and have two
pools?  One pool of active threads, one of sleeping threads.  Any time
a thread finds itself out of work, it just goes back to sleep.  When
the system decides it needs another thread, it just activates a thread
from the sleeping pool.  I believe JC was the first one to suggest
that model on this list.  :)

Another advantage of this model is that it lets you set a hard limit
on the number of threads.  Thus letting you avoid running into the
useful-#-of-threads limit your system will have.  Heck, it's a good
way to find that limit - just play with the number of total threads
and watch the system.  Eventually, you will find that adding a thread
to the pool only makes things worse.

If you were truely clever, you might find a way to let the system find
that limit automatically by letting the system add threads to the pool
and watch the response time.  I'm not sure exactly what metrics the
system would have to watch - total number of events processed over
time?  Average time to process an event?  I'm sure there are a few
that you could use to let the system figure out when adding a new
thread is just going to make things worse.

Heh.  Sounds like a prime candidate for thesis work... I wonder if
anyone's done it already?  *sigh* Off to search the web.  ;)

-Greg


_______________________________________________
MUD-Dev mailing list
MUD-Dev at kanga.nu
https://www.kanga.nu/lists/listinfo/mud-dev