Dynamic Loading of Modules

[Niklas Elmqvist]

On Thu, 19 Mar 1998, J C Lawrence wrote:
> On Tue, 24 Feb 1998 00:12:50 PST8PDT 
> Niklas Elmqvist<d97elm at dtek.chalmers.se> wrote:
> > As I said earlier, I am aiming to decouple the actual core server
> > from the world processing routines. Ideally, the main server would
> > not even *know* it is running a MUD, it would be sort of a
> > general-purpose core...
> 
> You have loosely described the general purpose network server espoused 
> by such as the ColdX project.  Cold is currently being used to server
> MUDs, web sites, as an SQL database frontend (for web) and otehr
> purposes.

Ahh, *that* is what Cold does. I've checked out cold.org a couple of times
but never really understood what it all was supposed to do -- it was so
very generic. Nice to see that one's thinking parallells that of other
people, though.

> > ...and all the functionality such as DB
> > management, socket I/O and AI would be implemented in free-standing
> > modules separate from the main executable (at least, I envisioned
> > this in the beginning). 
> 
> Which abstraction is deeper than Cold (or any other that I know of)
> has attempted.

Of course, this abstraction comes at a price. Since the modules are very
decoupled and don't know anything about each other's existence, I need a
decoupled way of allowing them to exchange information. I envision this
mechanism as a request chain on which the individual modules can put their
ear to and listen on (alternatively, register their interest in different
kinds of requests to a request manager or something). Whenever a request
comes along of the desired type, the module snags it and processes it
(this request may be a buffer of unprocessed text from the socket I/O
module, or a user command from the interpreter or whatever). The module
would then possibly emit a request on its own on the request chain to
pass along its processed information (a parser would emit parsed text, an
interpreter a user command, a magic module an area effect, etc). This way,
I have a decoupled intermodule communications chain. 

As I said earlier, the problem I can see with all this is that it is not
going to be very computationally efficient -- but it is neat, IMO. That
is, for every command issued by the user, for example, I will need a
potentially large number of requests passed between different modules: 
	socket I/O => parser => interpreter => user command handler
and so on. Also, I expect that using C++ instead of C (like I did in my
earlier effort PirateMUD) will also decrease performance somewhat,
although I have no idea about how much. No matter. It all just means I'm
going to need a bigger server. <cough>

> > I mentioned this to a RL friend of mine, and
> > he pointed out that it all sounded very much like conventional OS
> > architecture, especially like the module system of Linux kernels. 
> 
> I should point out that the original version of Shades was written to
> run on a DEC PDP-11.  In truth Hazeii hacked RMS to be the MUD (mainly 
> the command interpreter) so that effectively the OS _was_ the MUD.  

Sounds like fun. :) Hey, everybody, why don't we all take the Linux kernel
and modify it to become MudLinux! Or we could call it Mudix or
something...

> <bow> Cute.  Note that the vtable handling could easily get very
> messy, especially if you abstract heavily, go for multiple
> inheritance, or any of the nastier virtual structures. 

Well, the vtable handling is hidden from me by the compiler, so hopefully
I won't have to deal with that. I *do* have to deal with potential
performance loss, though, I expect.

> What platform/compiler BTW?

This on a x86 (Pentium II, as a matter of fact) running Linux 2.something
using the gcc/g++ 2.7.2.2 compiler... I've heard that gcc 2.8.0 and egcs
fixes some nasty problems in gcc 2.7.2.2 (especially when it comes to
templates, the STL string class header will not compile if it is not the
first STL #include in a source file, for example), so I am thinking about
upgrading. Furthermore, I have no idea whether the dynamic linking scheme
works on other platforms than Linux -- it *should* work on SunOS and
Solaris (and possibly other Unices) since the dl*() functions are derived
from Sun.

> > Incidentally, a
> > friend (the one mentioned above) later reported that this is
> > impossible in Windows-style DLLs.
> 
> Yup.  The features behind this working are that the name mangling and
> vtable manipulations remained consistent.

I actually ran into trouble with name mangling when I at first attached a
module to the core binary and tried to call a factory function called (not
surprisingly) foo() in the module which would return a bClass (a class 
secretly inherited from aClass) object. Of course, g++ had mangled foo()
into something like foo_fV (I discovered this with the nm utility). By
instead doing
	extern "C" aClass *foo(void) 
I prevented this name mangling.

> > If the base classes are correctly
> > implemented, the core will never even have to be recompiled and thus
> > never rebooted (not accounting for fatal bugs, that is <g>)!
> 
> You're going to need a very clean connect/disconnect model (and likely
> require single-threading unless you want to be excessively
> entertained) to ensure that no code points are in or about to enter a
> modeule which you wish to remove, or retain state data, or
> back-references (such as instances) for the about-to-be-removed   
> module.

Haven't thought of this, but I was probably going to implement this
single-threaded nonetheless. As for the connect/disconnect model, this
will probably be handled by a special module handler which is part of the
core functionality of the server executable. This handler would ensure
that all references to a module are cleanly removed when disconnecting it.
Because of the decoupled module architecture, the core server is the
*only* part of the MUd which knows about a specific module and its
methods, and it only keeps a reference and knows the base class of the
module at that (which could be a class called BaseModule or something).
All data flow is handled through the request chain mentioned above, as
well as through events in the event list. Therefore, I would never have
any problems of having another module keeping invalid references to a
detached module -- the only effect of a detached module would be that it
would no longer catch requests on the request chain (possibly leading to
the request being unhandled, which of course would result in a system
error or something like that, but would not hang the server). 

> J C Lawrence                               Internet: claw at null.net 
> (Contractor)                               Internet: coder at ibm.net
> ---------(*)                     Internet: claw at under.engr.sgi.com
> ...Honourary Member of Clan McFud -- Teamer's Avenging Monolith...

-- Niklas Elmqvist (d97elm at dtek.chalmers.se) ----------------------
"You can't trample infidels when you're a tortoise. I mean, all you 
 could do is give them a meaningful look."	 
	- Terry Pratchett, Small Gods