[MUD-Dev] Re: TECH: Distributed Muds

[J C Lawrence]

On Fri, 27 Apr 2001 08:48:02 -0600 
Chris Gray <cg at ami-cg.GraySage.COM> wrote:

>> From: J C Lawrence <claw at 2wire.com> 

>> I've never liked the base concept of a select() looping process
>> that tried to do real work between select() calls.  It can work
>> to be sure, but it has always smelled of playing with fire.

> Can you mention any specific concerns? You have to do a select()
> regularly in order to find out which client sockets have activity.

The basic problem is ensuring that you can return to the select() in
good time given a variously complex and often pathalogical softcode
world.  It can be done, it has been done, but it enforces and
requires constraints which I don't like either stylistically, or in
terms of what it forces my softcode engine to do.

> (I use it to know when I can *send* to them as well as receive - I
> mark all of the sockets as non-blocking, but only add them to the
> select write-set when they have plugged up.) If you don't use
> select(), then it seems you either have to poll all of the
> connections, or have a thread per connection. 

Nope.  I use the same select() basis you refer to, its just that it
sits in a mostly idle/blocking loop which the game, and in
particular the softcode executes in another context.

> Neither seems terribly efficient to me. At work we've been running
> 50+ threads on Linux and it works, at least until you try to use
> gdb with them, but it still seems dicey to me.

Unfortunately in the earlier days of this list I went rather thread
mad, and advocated others to do likewise (blish).  I'm currently
taking Cool and hacking the tolerable bits of Murkle ontop of it
with an eye to eventual release or real play.  While some of the
finer details haven't been resolved in terms of the process model,
the basic idea is that a well saturated busy server will sit in the
8 - 12 range in terms of threads (contention rates get too high past
that).

The other goal is to then play with this thing under Mosix...

>> I prefer having a thread dedicated to socket IO, pulling in and
>> buffering the IO as received, and then handling compleated blocks
>> off to one or more queues (depends on whether you do pre-sorts)
>> for the other game systems to pick up and process appropriately.
>> There are several advantages, not the least of which is that you
>> decouple your command/entry model from your processing model.

> That might be conceptually simpler to you, but it strikes me it
> adds issues with locks and shared variables. 

Nope.  On the input side the structure is simple:

  THere's an array of sockets, each of which is associated with a
  command buffer.

  A loop spinning on select() builds the command buffers.

  When a command buffer is full, it is detached from the socket and
  stuck onto the end of a FIFO.  A new command buffer (local pool)
  is then attached to the socket.

  The other end of the FIFO sits in the game thread (singly linked
  list), with the thread just popping command buffers off the end.  

  Processed command buffers are handed back to the memory pool..

On the output side there's a FIFO coming from the game thread
heading for the IO thread.  The handling is basically identical to
the above.

Contention points (ie shared variables and their implicit race
conditions):

  The FIFO structure is a single linked list managed by a
  structure which maintains head and tail pointers.  

  Each FIFO has a contention point across the relative state
  maintained by the head and tail pointers, especially as regards
  the when the list is emptied.  

  The list pointers between nodes have no contentions as they can be
  handled by ordering the pointer assignments correctly.

There are three FIFOs (in, out, heap), so there are three mutexes.
I handle these with counting semaphores.

No other shared data.  No other shared variables.  To keep this
true, I do end up passing some state in-band (specially formatted
messages) down the appropriate FIFOs to handle teardown and
throttling, but that's a minor complexity.

> In any case, it seems less efficient, and this discussion has been
> a lot about the efficiency of things. I'm a strong believer in
> keeping things simple until I really know, based on direct
> evidence, that I have to go to something more complex.

Aye, there's value in that.

--
J C Lawrence                                       claw at kanga.nu
---------(*)                          http://www.kanga.nu/~claw/
--=| A man is as sane as he is dangerous to his environment |=--
_______________________________________________
MUD-Dev mailing list
MUD-Dev at kanga.nu
https://www.kanga.nu/lists/listinfo/mud-dev