[MUD-Dev] Re: DevMUD - thoughts.1

[James Wilson]

On Sun, 25 Oct 1998, Chris Gray wrote:

>Adding things at run-time (new properties ("class members"), new
>actions ("class methods")) is not incompatible with a strongly typed
>language. You just have to have the right view of things, and not
>be glued to the conventions that C++ and Java use. My MUD language
>is strongly typed, but I can do this at run time (in wizard mode):
>(Warning: 'private' does not mean what it does in C++ - it just says
>who can see the newly defined *symbol*.)

yes, that's the other meaning of 'dynamic typing' that I was advocating.

>I don't disagree. In my system, the very nature of the parser requires
>going from compiled code to interpreted code to compiled code to
>intepreted code (and possibly more) to handle nearly any input. What I'm
>trying to avoid is requiring extra overhead for native => native calls
>across module boundaries.

okay. that makes sense.

> >I'm trying to understand, so let me ask if this example is something along
> >your lines:
> >
> >Module 'web_support' is set up to work using an external resource 
> >'g_net' which is some subclass of interface 'net_manager'. When 'web_support'
> >is loaded, it needs to find 'g_net' (i.e. resolve its external reference).
> >It asks some broker to give it 'g_net'. (There can be multiple specialized
> >brokers, or one huge one.) The broker determines if 'g_net' is currently 
> >available, and if not tries to find a module which claims to provide it. 
> >Assuming said module is found, it is loaded and the symbol is provided to
> >'web_support'.
>
>That's a sort-of valid example, I think. I hadn't thought about the idea
>of somehow automatically loading modules to resolve undefined references. I
>was more concentrating on how explictly loaded modules would be tied
>together. If we use 'dlopen' for loading, would a load module needed
>by another be automatically loaded by dlopen? How would *our* loader know
>that a given module exports a 'g_net' before someone has loaded that
>module?

modules would need to register their functionality with the dynamic loader,
so they could be loaded when such functionality is requested. cf. kerneld,
which requires you to run depmod first.

>I had thought that we wanted the various modules to be compiled completely
>independently of one-another. The only header file I had envisioned for
>inter-module stuff was one that defined constants for all of the currently
>existing interface kinds, and provides prototypes for that kind.

see the end of the message for more on this.

> >Using a
> >straight function call doesn't allow someone to write a handler which 
> >has its own individual state, e.g.
> >
> >static string s_magic;
> >
> >void inputHandler (/* rep */)
> >{
> >	// manipulate s_magic in a non-reentrant way
> >}
> >
> >is BAD in a multi-threaded environment (and inflexible in any case). The
> >alternative is quite appealing:
> >
> >struct input_handler
> >{
> >	virtual void handle (/* rep */) = 0;
> >};
> >
> >can be specialized to be thread-safe and have any state it likes:
> >
> >struct my_input_handler
> >{
> >	virtual void handle (/* rep */) 
> >	{ /* magic */ }
> >private:
> >	string _magic;
> >	mutex _lock;
> >};
>
>I understand the C++ syntax you have written (except for "/* rep */" - is
>that just meant to copy the prototype?), but I'm completely failing to
>see what you are getting at. Sorry.

Writing it as a straight function call makes it impossible to extend by
subclassing, whereas using an object makes it possible. Since we don't
necessarily know all the things we are going to want to do in two months, 
four months, a year, why limit ourselves by choosing the more inflexible
option?    


persistent
>virtual world, all state should either be attached to the entity
>(e.g. player character) doing the action, or be global state attached to
>some other persistent entity in the virtual world.

*blink* why is that inconsistent with a handler having state? suppose I
wanted to write a persistent object that WAS an inputHandler? or do you 
mean REALLY global state?

>In either case, the
>through the system's database.

this would assume that there's a distinction between DB objects and in-memory
objects, no? that is an issue which AFAIK hasn't yet been raised or resolved.

> There should be very little static state in
>any modules, and that all has to be protected by mutex's or spinlocks. Or,
>the module could use queuing to serialize accesses. An example of static
>state would
>maintain.

agreed, though I would again write that as fields in some object rather than
as static, so one could have multiple telnet servers running simultaneously.

> >IMO, interfaces should be up to the module programmers. Think of a module
> >as a shared library; to use that shared library, you write a program that
> >#includes the proper headers, and link it against libfoo.so. It's up to
> >you to use libfoo's api according to the headers, and up to the compiler
> >to check your errors. If you want to write a new module with some funky
> >interface, and Bob wants to use said funky interface, that should be as
> >easy as #including your headers and linking against your library (or 
> >whatever analogue applies).
>
>If some modules wish to use shared libraries, that is up to them. 'dlopen'
>can take care of that. It is outside our concern. What I think we need is

no no, the analogy is that modules ARE shared libraries.

>ways whereby, e.g. modules that provide user input/output to the system
>can be linked up to modules that provide parsing, scenario code, and
>database operations. That linking up isn't done by name, it is done
>based on the semantic nature of what is needed.

how is this semantic nature determined? 

> Hmm. I suppose you could
>use names to indicate that semantic nature. E.g. any module that exported
>a parser would export a public symbol called 'parse', and the dlsym
>facilities could be used to find that at run-time, and connect to it
>from an input provider. However, that is just replacing the set of
>constants in a header file with a set of strings, which would likely
>also be in the header file beside the prototypes needed.

some unix systems will do this automatically (I believe Linux
does) while others require you to roll your own inter-module
linking. rolling your own is thus more portable, and also much 
more amenable to nice warning messages and graceful recovery 
from link errors. ("Unresolved symbol _Rxsv1_void_foo_1Qvoid", 
anyone?)

>In this scheme of the header file containing the prototypes for the
>interface kinds, it is quite easy to add new interface kinds. By
>definition, no module is using or exporting an interface kind before
>that interface kind is defined. OK, so someone needs a new one. They
>add it to the big header file, along with the prototype they intend for
>it. New modules can now be written that communicate using that new
>interface kind. Old modules, which by definition do not use the new
>kind, are not affected. Similar if someone wants old modules to use
>the new kind - they must be modified to use it, and all is still well.
>WOLOG (one of my math profs used that: WithOut Loss Of Generality).
>
>There is no particular reason that the "one big header file" couldn't
>be split up into smaller pieces, if there are specialized interfaces
>that only a couple of modules need. However, there does need to be
>some mechanism (Perl scripts?) to check that no interface kind ID
>(whether a number or a name) is used more than once. We don't want
>a parser calling a database's 'fixup' interface when it is supposed
>to be calling an output filter's 'fixup' interface.

this is not an issue with C++ or suitably name-mangled C.

>James, it seems to me you are thinking of building modules that are
>aware of each other at compile time. I've been trying to avoid that,
>so that only at run-time are they actually attached to one-another.
>But then, I could be missing your point altogether. How about a
>more detailed example?

I guess I don't see the alternative to having modules know about one
another at compile time. Either they refer to some external symbols
(which end up being function pointers or objects with virtual methods)
or ... what? No module refers to any external symbols? All function 
calls are done by (a stringified) name?

Here's what worked for me a couple of years ago.

module web_magic contains functions which use network functions such as
gethostbyname. These network functions live in another module, 'net_lib'.
When web_magic is written, something is put at the top which says "I 
depend on net_lib". The C++ code for web_magic contains module initialization
code (_init () in most dlopen implementations) which is responsible for
dynamically loading net_lib  and linking up the functions web_magic uses 
with their bodies in the net_lib module. So, this could look like

<pseudocode>

/* web_magic */

/* some definitions here of function pointers or polymorphic objects
 * which we need from other modules, e.g. */

static struct hostent *(*_gethostbyname) (string name);

extern "C" void _init (void)
{
	/* use the dynamic loader to pull in the external
	 * references we need. Perhaps we name netlib.so explicitly */
	void *net_lib = dlopen ("net_lib.so", RTLD_NOW);
	_gethostbyname = dlsym (net_lib, "gethostbyname");

	/* or maybe there's something like kerneld that will find the
	 * netlib module for us and give us the right reference: */
	_gethostbyname = dynaloader->resolve_function ("gethostbyname");

	/* or maybe there's even a type-safe loading mechanism */
	_gethostbyname = typesafe_dynaloader->resolve_function 
		("struct hostent *gethostbyname (string)");
}

</pseudocode>

So, 'web_magic' is _aware_ of net_lib at compile time, but is only 
linked up with it at load-time.

This is all off the top of my head, BUT something like it did work.
Actually, it was wrapped up in some C++ static initializers so 
nobody'd ever have to care about the minutia of dynamic loading on
Win32 versus Irix versus Linux. 

>Please, someone jump on me if I'm coming on too strong here. I'm not
>trying to squelch anyone else's ideas and proposals. I'm just espousing
>some thoughts I've had.

you're not coming on strong, at least in this message. and besides, what's
wrong with a little debate? we're not delicate little flowers.

James