[MUD-Dev] Embedded languages, object persistance... ack.

[cg at ami-cg.GraySage.Edmonton.AB.CA]

[Joe Kingry:]

[The old-timers on the list would likely have guessed that this is the
kind of question that would get a response from me, since I'm known to
be a programming-language geek!]

> I want a mud server on which I can change the code online of say a skill, a
> spell or something else. I understand that this is the purpose of having an
> embedded language that runs "on top of" the mud's driver. i.e. MUF for MUQ,
> Python for AR3 etc.  So in general I know this code is to be compiled since
> you don't want to be executing "scripts", but is this code an object? and
> then is the byte-code stored? ack.. I'm confused.

As previous responders have indicated, note carefully that what I say
here is what *I* think, and by no means the best or only answer. You
can read far more than you want about my system at:

    http://www.graysage.edmonton.ab.ca/cg/MUD/AmigaMUD

(I still call it AmigaMUD even though I'm totally on Linux now - I need
a new name, updated images, updated docs, etc.).

There have been technical papers on the idea of a "persistent programming
language" for several years now. The earliest I know of is APL, which used
"workspaces" on IBM mainframes in the 70's. The basic idea is that if you
write an assignment statement like:

    name := "Fred";

then the variable 'name' will keep that value over reruns of the system,
and even reboots of the computer. Given that, and more complex types in
the language, you can produce a distinctly different style of programming
environment. Such a programming language can be extremely useful in a
MUD system, especially if the language and its environment is also customized
for MUD-type activities.

Note that the above viewpoint is a very programming-language-centric one.
The main concern here is the MUD, and a persistent programming language is
just one possible tool for building the MUD. From a language point of
view, the alternative to being a persistent language is to require
explicit I/O calls of some kind in order to achieve the persistence.
That certainly works, but can be more of a pain to program in.

Is there such a language, and persistence system (whether it be an actual
database or something written specifically for that language system)
freely available? I dunno. Sorry about that. Since half the fun of doing
my MUD system was writing the persistent language, I wasn't motivated to
go looking!

Does such a language have to be a lot slower than other "more standard"
languages? It depends. There is overhead in being persistent, but that
overhead can be less than other overheads that are possible in the choice
of language features. So, there can be persistent languages that are
faster than many (perhaps even most) non-persistent languages.

> All I want in a mud server is this:

> An added bonus would be, but hardly nessesary:
> -driver ported to both Win32 platform and *nix platforms and database is
> independent of platform.

My system does all of those except the HTTP access. I'm not sure what
that would mean in many cases. How do I convert a list of references to
bytecode streams into something to display on a web page? I haven't
actually ported my main server to Windows, but the port wouldn't be
very hard. If you stay away from any GUI or MFC things, and stick to
the non-asynchronous socket stuff, you should be mostly OK. Oh, and
don't use Windows threads or Windows messages, either.

How does it all work and fit togother? I went looking at my online docs
a bit, and nothing explains it too clearly, so I'll give it a go here.
I'll start at the lowest levels and work up from there.

My "database" was build for the MUD system. It does not store traditional
records or tables at all. Instead, it stores "characters", "grammars",
"things", "actions", "strings", "lists", "tables", etc. These are of varying
lengths, and some, like "lists" and "things", change their lengths at times.
Each entity in the database has a unique 32 bit key. Part of that key
is a direct index into the contents of the "MUD.index" file. That yields
a short descriptor of the entity, giving the size of the slot it currently
occupies, the amount of space it currently uses, and the offset of the
actual data in the "MUD.data" file. This level of indirection allows the
database layer to move entities around in the file as they grow and
shrink, although it tries to minimize such movement.

Currently, the entire "MUD.index" file is kept in memory. Some portion
of the MUD.data contents is kept in an in-memory cache. Entries in
that cache can also grow/shrink and move around. The cache is "write-back"
in that changes made to cached objects are not written to disk right
away. This reduces the disk I/O tremendously. The MUD server also uses
others kinds and levels of caches to reduce that I/O even more.

So, lets say a player character called "Fred" is logging in to the game.
A connection comes in, containing the string "Fred". The server has a
few pre-defined entities in the database, having fixed keys. One is the
player table, which is a string-keyed hash table, the values of which
are further database keys. This table is stored in the database, like
everything else. The server requests the player table from the database
layer, and gets back a pointer to an in-memory copy of it. The server
looks up "Fred", and gets back a database key to the "player" structure
for player "Fred". That contains the encrypted player password, various
various flags, and various other database keys. These player structures,
like everything else, are scattered all over the database, although each
individual structure is stored as one contiguous entity.

Assume the password is verified, and game-play is to start. One of the
keys in a player structure is the database key of an action (written
in my persisent MUD language) which is to be executed when that player
reconnects to the MUD. The server requests that action from the
action cache, which may in turn have to fetch it from the database
layer, which may in turn have to read it from disk. Eventually, it gets
back a pointer to an in-memory structure describing the action, and
having pointers to one of the two possible executable forms of it (I
originally used a "parse-tree" executable form, and now also have a
somewhat faster byte-code form as well). The server calls the code.

That reconnect code can in turn call on other MUD-language functions and
server builtin functions. This activity can result in messages to the
player, messages to other players nearby, graphics changes to some of
those (my MUD is multimedia), changes to persistent values, etc. The
function eventually terminates, and Fred is fully logged in.

Similarly, there will be a key in Fred's player record that points to
the function to be used to parse and interpret commands coming from
the player. There are other, similar, keys pointing to functions to
handle mouse clicks, numeric keypad typing, etc.

Let's say Fred does something to change his simple string description.
Buried somewhere in the MUD-language code which interprets user commands
will be a simple assignment statement like:

    Me()@p_pDescription := newDescription;

"Me" is a builtin function (implemented directly in the server) that
returns the database key for the active player. The '@' operator means
to select a property in a "thing". The whole of the above means to
assign a new value to that property on that "thing". In my system,
such properties are not fixed, like in a C++ class. So, doing the
above assignment may cause the structure for Fred's "thing" to grow
in size, if it didn't previously have a "p_pDescription" property. The
system takes care of that, and finds a place in the database to store
the string from variable "newDescription". The database key for that is
added to Fred's set of property-value pairs (properties are just database
keys too), via the pointer into the database cache that came from the Me()
key. That database entity is marked as "dirty" because of this, and will
eventually get written to disk by the database code.

One of the keys to all of this is that the interpreter/bytecode-engine
in my system knows implicitly about the database, and how to talk to
it. E.g. The piece of code:

    thingy at EggCount := thingy at EggCount + 1;

translates into bytecode something like:

    pshl    thingy	; access local variable
    pshref  EggCount	; push a fixed 32 bit database key
    pshl    thingy
    pshref  EggCount
    getPrp
    add1
    putPrp

The bytecode machine knows that to execute a 'getPrp' instruction, it
takes a pair of database keys off of the bytecode machine's stack, and
passes them to an internal routine 'getProp'. That routine in turn
retrieves both entities from the database, and looks for the "property"
one in the list of properties on the "thing" one. The resulting value
is then pushed onto the bytecode machine's stack. Similarly, the
'putPrp' instruction passes the top 3 stack elements to 'putProp', which
adds that "property" to the "thing". That may involve a call to database
routine "ioExpand" to expand the space for the "thing", or a call to
"ioDirty" to mark the changed "thing" as needing to be flushed to disk.


Hmm. This didn't come out like I had imagined. Hopefully its useful.
Feel free to ask more questions if you want more details.


To answer some of your specific points:

> -stores things in a database file(s). No more 4000 player files.

Yep.

> -allows me to define objects online. Ie. I can decide I need an object that
> will act like a boat, and I can code and modify this online.

Yep. I have a small program, "MUDCre" that creates a nearly empty database.
*Everything* else is built by sourcing MUD language source files to
create stuff. Note that is likely a very bad idea to do all of your
building online, unless you have a very good way of dumping that back
out in a form that you can then feed back in. For some of the trickier
stuff, it is easier to look at things in a nice editor, and to be able
to use the edit/compile/debug cycle, than to try to get it all correct
online.

> -doesn't have every object in memory, only when needed, otherwise on file.

Yep.

> -doesn't require me to learn it's "own custom embedded language", this one
> really gets to me

Sorry, but I have a custom language. I view this as a good thing. Sure,
it takes a bit of time to implement, but its a fun thing to do (well, it
is for programming-language geeks like me). The custom language is also
likely to be much more appropriate for writing MUD scenarios that some
randomly chosen language. In particular, it won't have nasty things like
uncontrolled ways to issue system commands, write to disk files, etc.!

> -somehow allows for security and access levels while coding

Yes. See my online docs.

> -somehow allows for an event type system

No really a separate system. See my 'After' builtin routine, and how
handler actions are attached to player and non-player characters.

> -allows code to be contained in packages and has appropriate management
> features. Ie. I can write the code offline in a file and then upload the
> file to the server

Yes, except that I use the concept of "symbol tables" to encapsulate
things. I find it works quite readily, and by hiding symbols in private
tables, only the owner of those symbols even knows they exist.

> -will allow for http access to the database

Nope. I have no idea what this would mean, in general.

--
Don't design inefficiency in - it'll happen in the implementation.

Chris Gray     cg at ami-cg.GraySage.Edmonton.AB.CA
               http://www.GraySage.Edmonton.AB.CA/cg/



_______________________________________________
MUD-Dev maillist  -  MUD-Dev at kanga.nu
http://www.kanga.nu/lists/listinfo/mud-dev