[MUD-Dev] Gamora: Lessons learned? (fwd)

[Niklas Elmqvist]

Here is the second reply from Scott. Please ignore any acid remarks from
me about the list not accepting my suggestions :)

-- Niklas Elmqvist (d97elm at dtek.chalmers.se) ----------------------
  "The trouble with being a god is that you've got no one to 
   pray to."
		-- Terry Pratchett, Small Gods

---------- Forwarded message ----------
Date: Tue, 27 Oct 1998 17:08:00 -0500 (EST)
From: Scott Miller <scgmille at indiana.edu>
To: Niklas Elmqvist <d97elm at dtek.chalmers.se>
Cc: gamora at ntlug.org
Subject: Re: Gamora: Lessons learned?

> stupid :)
> 
> 1. Does Gamora provide any other basic primitives than inter-plugin
> 	communication using buses, and threading? Distribution?
The buses are there mainly for non-directed messages, you can communicate
to any BusObject using a method call on that BusObject, or by having the
bus route it to that bus object (which is still just method calls, so its
blazingly fast).  Gamora supports transparent distribution.  MMG
(Multi-machine Gamora) constructs virtual Busobjects that look just like
the real thing, except that the real object exists on another Gamora.  You
communicate with it the same way, so to the application, the distributed
aspect is transparent.

> 2. I thought I saw something about writing plugins in C? Is this possible?
Sure, just use the Java Native Interface, to write code that supports the
BusObject methods, and Gamora will have no problem with it.

> 3. What are the "security features" of Gamora? That the controlling object
> 	(forgot your term for it) has a higher priority rating than the
> 	rest of the plugins so that it can kill any malicious plugins?
> 	Anything else?
Actually, not quite.  The BusController is just a BusObject that gets
non-directed messages on the bus it rests on.  The Kernel, whose
documentation I agree is far to technical for someone with no Gamora
background, does the thread management.  All threads come from the Kernel,
and it can keep an eye on the threads to make sure no thread monopolizes
system time.  This happens automatically, and isn't a concern when
writing plugins.  As far as protecting from malicious plugins, you have to
understand that there isn't any way to introduce a plugin into gamora,
unless you have a plugin running that can load new plugins.  Currently,
there are two such plugins, GamoraFilesystem, which provides a shell-like
interface to all aspects of a running gamora, and another which isn't
loaded into Gamora by default.  
   Each bus has a set of required security levels to perform some
operation, Add, Remove, Add a controller, etc.  If the security level of
the Bus (each BusObject has a security level) is lower than the required
security level for an operation, that operation is denied.  If a BusObject
has a higher security level than the bus on which its trying to do
something, it can override these to do the operation.  So, if all Buses
are locked from doing Add, for example, and no BusObject has a security 
level high enough to override, no BusObjects can be added.  Its a lot more
flexible than that of course, but thats the basic idea.

> Uh-huh... The discussion goes something like this on MUD-Dev as well. 
> We're thinking in terms of network I/O passing messages to a lexer, then a
> parser, which then turns the input into commands executed by the
> MUD-language softcode or by native code modules. Of course, this is for
> text-MUDs only, so the module interdependencies might look very different
> in a graphical setup of DevMUD.
Right, this would all be very easy in gamora, because you'd likely lay out
each of these levels on their own bus, and pass the input downward,
process it at a layer, pass it upward where its processed by that layer,
etc.  You could simply change out the BusObjects on any layer to affect
behaviour without changing a single line of code in any other layer.  You
could even do this in real time without recompiling anything.

> I see that Gamora seems to provide some basic functionality to make
> writing server applications easier, that is, primitives which the
> plugins may depend on and use... (I asked about this above as well.) At a
> glance, I see the bus message-passing system, threading and security as
> some of these primitives. What more are you supporting? Are these
> primitives sufficient for your purposes, or are you planning/wanting to
> add more without loss of generality?
There are a few more, the networked BusObjects themselves, BusObjects that
store data (for an object-store), the inter-gamora stuff, authentication
bits.  Gamora is flexible enough to support anything else you want.  For
example, someone in the Gamora team is writing (has written) a Web server.
He wanted logging facilities, so he is writing a BusObject that is a front
end to a log file.  If you want a new logfile, add a new instance of the
LogFile BusObject.  He has them all on one bus, and all as controllers, so
he can write to multiple logfiles by simply passing one message to the Log
bus.

> FYI, in the DevMUD project, which is less generic than Gamora since it
> concentrates on networked game servers, I am proposing a "core" which
> provides the following primitives:
> 	- Event manager (all games need one)
> 	- Module manager (takes care of dynamic loading/unloading)
> 	- Inter-module communication mechanism (message passing between
> 		modules)
> 	- Multi-threading functionality
> 
> The point of DevMUD is to provide a framework which MUD developers can use
> to build their games on. Therefore, it is our duty to provide some *very*
> useful and powerful primitives to make using the platform worthwhile;
> otherwise, programmers will keep on rolling their own and working from
> scratch. I suspect you've had to think about this as well; if Gamora does
> not *add* anything special (i.e. raises the level of abstraction and makes
> life easier for developers) to general server development, then who will
> use it?
Well, to be honest, the extra special stuff was in there from the very
beginning.   The netorking modules, that provide say Text based
communication over TCP/IP, were the main attracting part to Gamora.  Never
again would you have to write comm routines.  You just ask for a TextNode
to "foo.com", and voila, you start getting messages over the network.
  I suspect the same kind of thing could be done in the MUD system.  For
example, a very generalized database for storing worlds, so you don't have
to worry about file formats and data access anymore, just ask for a
WorldFile named "world.dat" and you get a well abstracter interface to it.

> Btw, I was also proposing a bus-based message system very reminiscient of
> Gamora's (though I called them "pipes"), but this was severely attacked
> due to efficiency reasons. :( You would not have any performance numbers
> of Gamora, would you (especially about the additional overhead a bus-based
> message architecture imposes)? I realize that Gamora is not aimed
> primarily at performance (it is in Java, too, not in C/C++ like DevMUD),
> but it would nevertheless be interesting. 
On my machine, I've clocked the bus system at passing 500,000+ messages
per second.  Keep in mind, the message size does not matter, as we're
basically passing around pointers to data, so that could be large sums of
data.  Gamora speed isn't limited in the sense you may think, because the
data itself doesn't have to be moved (unless its to go to another Gamora).

> On a similar note, I guess that none of the plugins in a standard Gamora
> system knows of each other when they are loaded into the system? Do you
> have a standard way of having two plugins "shake hands" with each other
> and initiate communication (such as creating a bus between them -- I have
> no idea) (that is, a protocol of sorts), or is this done manually or
> through the use of config files (that is, the config of this plugin says
> that it should make contact with this and that module...). Is there a
> broadcast channel for newly loaded plugins to announce their presence and
> offer their services? How do plugins advertise their capabilities? Are
> plugins designed to work with each other? 
If you needed this kind of functionality, its available.  You can "look"
on the buses to see if Objects are there.  Also, signals are generated on
a bus each time a new BusObject is connected or removed, so if you want 
to know if a plugin gets loaded (or a new connection is formed, or....)
you can stick a Controller on that Bus which will get these signals, then 
you negotiate however you like.  
 
> Or maybe all this is left to the guy building the plugins? But this would
> thus ensure that all plugins would be built together as a suite, which
> kind of defeats the purpose of a plugin. A plugin should be possible to
> add to any system and just jump right in and do its stuff. 
Well, the definition of a Plugin in Gamora, is anything that can be loaded
without knowing how to construct it.  An application in gamora is
typically a plugin that constructs a number of buses and BusObjects on
startup.  For your project, for example, you might have "AI module 1", and
you code your plugin to create the MUD server bus, add that AI module
(which is a BusObject), and some other stuff (world reader, connection
listener (provided by Gamora)), etc.  If at a later date somebody comes up
with AI Module 2, you just change your plugin to load that one instead.

> Hmm, and Gamora is *not* a server builder? It strikes me as a very generic
> and flexible server abstraction. Is there an even higher abstraction I am
> missing?
It works VERY WELL as a server builder, but it can be other things too.
The Casbah project, for example, is building an application framework with
language agnosticism, data agnosticism, and some other really interesting
goals.  One of the proposed uses for Gamora there is as an Object-store
cache, where Gamora sits between casbah and many different kinds of
databases, caching data as its read.  Due to the distributed capabilities
of Gamora, this cache is shared with other Casbah's transparently, data is
accessed by other Casbah's through a unified cache.

> Btw, do you mind if I post any of your replies to the MUD-Dev mailing
> list? Gamora and you will of course receive proper input credit in the
> DevMUD project contributor's list. :)
Go for it.  I also suggest reading the two IRC sessions that took place
about Gamora, they were good introductions to new users.  Also on the
gamora site: http://php.indiana.edu/~scgmille/gamora.html.

	Scott

--------------------------------------------------------------------------
 Java Programmer          Scott Gregory Miller          Linux afficionado
 Graphics Designer        scgmille at indiana.edu              General Loony
--------------------------------------------------------------------------