[MUD-Dev] Muq update

[Cynbe ru Taren]

On Wed, 29 Dec 1999 01:03:00 -0800 "Joe Kingry" <jkingry at uwaterloo.ca> wrote:

| Muq seems to have most of these except for MUF, I got tired of programming
| backwards ;).

*grin*

Me too, actually!

Muq has "always" had the design goal of supporting application languages,
but in the last couple of releases I've actually started implementing
this.

MUF (Multi-User Forth) is still the best-developed Muq softcode language, and
MUL (Multi-User Lisp) is still in limbo waiting for user demand, but we now have
MUC (Multi-User C) coming along pretty quickly.

I wrote MUC over a weekend or so because I was testing the Muq OpenGL bindings
and when it comes to 3D graphics, I'm too much of a C chauvinist to be willing
to write heaps of OpenGL test code in anything else.

MUC now implements all the basic C control structures except 'switch' (which
I expect to come along very shortly) and 'goto' (might take a bit longer).

MUC also provides some Perl4 level regular expression syntax and such.

Muq provides lots of functionality beyond the basic C model, most of which
is not yet available in MUC.

I expect to add in the needed hooks a few at a time over roughly the
next year, after which MUC will probably be the primary Muq softcoding
language.

But never the only one!  I expect and intend Muq to support a wide
variety of softcoding languages from the conventional and general
(MUF, MUL, MUC) to the specialized (I've barely started SQL) to the
outre' (APL, Prolog and Smalltalk would each be a kick to have
around for occasional use, in their varied ways) to the purely
experimental (why not HTML as a source syntax for programming?)



As long as I'm on the horn here, a quick Muq status report:

First beta (version 0.0.0) was released 99Dec29 as promised about
year go:  All future Muq server and library releases will be
upward compatible from here on out.

The Muq worldkit "oldmud" and distributed world "Micronesia" are
-not- yet in beta:  They are big projects in their own right, and
trying to beta them simultaneously with the Muq server proper was
(I quickly realized) not a realistic proposition.  I'm informally
projecting a Brigadoon Day premier for them, but that is not yet
a promise -- I have to see how much spare time Cisco is going to
leave me before I can schedule that rationally.

Muq source:  ftp://ftp.cistron.nl/pub/people/cynbe/muq.0.0.0-src.tar.gz
Muq docs:   http://muq.org/~cynbe/muq/muq.html

(Updating the docs is a high near-term priority.)

Muq mailing list subscriptions (manually maintained at present):

 muq-announce-request at muq.org     New releases
 muq-admin-request at muq.org        For administratore
 muf-coder-request at muq.org        For MUF softcoders
 muc-coder-request at muq.org        For MUC softcoders
 muq-coder-request at muq.org        For server hackers
 muq-user-request at muq.org         For end users
 muq-bugs-request at muq.org         For bug reports

All are deathly quiet at present;  I expect traffic to ramp up
rather slowly over the next year or so.



I'll close with a return to Joe's server laundry list, seen from
(surprise?) the Muq perspective:

| I think I have my object/class/code/driver concepts all muddled.
| 
| All I want in a mud server is this:
| 
| -stores things in a database file(s). No more 4000 player files.

With Muq I'm trying to push the envelope on nice handling of
persistence.

This means partly little touches like support for logarithmic backup
sequences, so that a scattering of older dbs are preserved as well as
the last N.

It also means dividing the full server state up into lots of logically
independent sections which I call 'db's -- in general, one per system
library and one per user, allowing them to be individually installed,
removed, updated or rolled back.

I believe this will prove to be an enormous win over monolithic server
database files without such update possibilities.

Most of this support was written in 1999, and will undoubtedly need
some refining in the light of practical experience.

All of the above is unrelated to physical organization of the server
state in terms of hostfiles.  Currently the state is kept in a single
hostfile, but future releases are likely to support such things as
keeping system libraries in a separate read-only dbfile -- to avoid
wasting space by backing them up with user data, and to simplify
even further updating of standard softcode libraries.



| -allows me to define objects online. Ie. I can decide I need an object that
| will act like a boat, and I can code and modify this online.

Muq's OOP facilities are based on the CommonLisp Object System, the
most stable and general standard I could find.  In particular, CLOS
defines what should happen to existing instances of a class when the
class is redefined, something essential in a persistent programming
environment such as a mud, but absent from such standards as Java and
C++, which limit themselves to throw-away one-session datastructures.

Boats are actually an example I use in my tutorials. :)



| -doesn't have every object in memory, only when needed, otherwise on file.

Muq keeps objects on disk and can demand-page any object as needed
in a single read() (i.e., half the time of gdbm or such solutions)
while still efficiently supporting large numbers of small objects
-- the norm for mud-style systems.

A ram buffer (typically set to 8Meg, but resizable) caches recently
used objects, with an MMU-style hashtable for locating them quickly.
Overhead due to the hashtable is about 3% of CPU used by the server
-- thanks to careful efficiency tuning!

The main caveat on all this at the moment is that the current
garbage collector is a simple-minded classic xlisp style
mark-and-sweep, which will force all of the db into ram on
each garbage collect:  This means that if you have much dynamic
allocation going on, in practice your db currently has to fit in
the ram buffer.

I've designed a two-generation production garbage collector to
address this issue, with garbage collection of generation
zero done entirely in-ram and garbage collection of generation
one done incrementally:  The design work was done in December
and I expect implementation to take about 1-2 weekends, and
to happen sometime this year.



| -doesn't require me to learn it's "own custom embedded language", this one
| really gets to me

I used to feel rather strongly about this myself:  It is one reason
I picked CommonLisp as a model for Muq:  It is a well-known,
standardized language which can be sanely used in a sandbox
environment.  I still expect MUL to eventually be close enough
to CommonLisp as to make little practical difference for most
users.

I looked at supporting ANSI Forth, but it really just isn't suitable
for sandbox programming of virtual world stuff.

C and C++ are likewise simply flatly unsuitable for high-level
scripting sorts of applications.  I've tried to make MUC look
and feel as much like C as is reasonable in the Muq environment,
however, and I believe that is what the typical C fan really wants.
(Witness the success of languages like Tcl and Perl which look a
lot like C superficially while having in fact only the vaguest
semantic relationship to it.)

Perl is in some ways better, but really pretty lame when it comes
to a lot of OOP-related stuff.

In the end, application languages really have their own set of
design requirements, and trying to model them too closely on
other kinds of languages is usually going to be a mistake.

Using an off-the-shelf app language implementation can of course
be sensible, if a good one is available at design time.  When
Muq started in '93, Java and such really didn't exist... :)




| -somehow allows for security and access levels while coding

This covers a lot of design decisions!  I've thought hard about
this stuff in Muq.  In general, Muq executes compiled functions
with the privileges of the caller (not of the author/installer,
something which I believe to be a major security design mistake,
common though it be) but allows SUID-style assumption of author/
installer privileges via specific source code syntax where
desired, allowing controlled access to protected resources.



| -somehow allows for an event type system

When I first studied the CommonLisp Exception System spec, I thought
it one of the most baroque, overgrown monstrosities in the history
of committee software design.

I -still- think the document describing it is one of the most
pointlessly opaque documents in the software world, but I have
gradually come to regard the design as potentially a big step
forward in software design and implementation, perhaps on a
par with the OOP paradigm.

CLES essentially opens a marketplace in which software can post
help-wanted ads and negotiate needed services in a much higher-level
fashion than OOP allows, much as OOP does it in a much high level
fashion than simple function calls or tables allow.

CLES also opens up the execution of programs to make them inspectable
and controllable in a high-level sort of way suitable for general
GUI-driven browsing of code execution on a system, vs the current
state of the software art of programs which hang endlessly and
function as sealed black boxes beyond end-user inspection or control
(except as explicitly provided via special GUI hacking).

I'd like to take Muq in the direction of leveraging Muq's
existing CLES-style exception handling with traditional
style GUI event streams, in place of the usual simple-minded
callback paradigm:  I think this has great potential.





| -allows code to be contained in packages and has appropriate management
| features. Ie. I can write the code offline in a file and then upload the
| file to the server

Muq has "packages" at the source level ala CommonLisp, similar to the
concepts of the same name in Java or such.

These complement rather than competing with the above-described
lower-level "db" concept:  A Muq db can contain many packages,
and can be exported or imported as a cross-platform portable
binary hostfile.  "db"s are essentially a physical-layout notion,
while "packages" are a logical-layout notion.  Both are useful,
even critical in large systems.



| -will allow for http access to the database

Someone needs to write an httpd internal daemon for Muq.  I'm
hoping to stall on this until someone else gets impatient
enough to do it. :)

Not a very big or difficult project.



| An added bonus would be, but hardly nessesary:
| -driver ported to both Win32 platform and *nix platforms and database is
| independent of platform.

Muq runs 64-bits on all platforms, as future-proofing and to promote
smooth networked operation between WAN-distributed nodes running on
different host architectures.  (I timed this at slowing down CPU
intensive stuff by about 15% on 32-bit intell machines, which I
consider a very reasonable cost.)

The Muq binary db files use the host byte ordering in order to allow
mmapp()ing of the files into memory and direct access via C memory
reads: This means that the db files can exist in either big-endian or
little-endian formats.  However, the Muq server will automatically
convert as needed between the two, so normally no portability issues
will arise.  An exception would be if you wanted to NFS-share a single
read-only copy of a library between processors running different byte
sexes: This wouldn't work, separate big-endian and little-endian
copies would have to be maintained.

I've thought of porting the Muq server to Windows:  It shouldn't be
any harder than porting TinyMUCK or such to Windows, which has been
done.  I've tried to keep the code as portable as reasonably possible,
with I think good success.  (The 32- to 64-bit conversion of Muq took
much less than a month, for example, whereas it would likely be
virtually impossible for many comparable servers.  Clean use of types
made the difference.)

However, I do not plan to port to Windows myself:  I feel that if I
can get 10,000 to 100,000 Linux users of Muq, someone will step
forward and do the port for me.  And if Muq cannot get that level
of support on Linux, why bother inflicting it on the Windows world? :)



ad astra!
  Cynbe



_______________________________________________
MUD-Dev maillist  -  MUD-Dev at kanga.nu
http://www.kanga.nu/lists/listinfo/mud-dev