[MUD-Dev] SfD: Clientside Caching

[Nathan F Yospe]

On Tue, 17 Mar 1998, Jon A. Lambert wrote:

:On 16 Mar 98 at 17:20, Nathan F Yospe wrote:

:> SfD: Subject for Discussion. I decided to stir things up. It has
:> been slow. I'm going to start posting these every few weeks, or as
:> the impulse strikes me... just to get the juices pumping. Feel free
:> to tear it to pieces.

:Good. I'm running out of juices and in danger of posting something 
:offensive.  This looks like a much more productive topic. ;)   

Glad _someone_ responded, I was worried for a moment.

:> Today's topic: Storage of data for a mud on a permanent or temporary
:> basis. For now, I will assume a system that only works with a custom
:> client, where 'client' means either a remote pure client, or a
:> 'smart' client, like those employed (presumably) by UOL. (Is M59 a
:> pure client? I'm pretty sure Pueblo is.)
:
:> A little background: 
:[Excuse my active snippage.]

No problem, that's what keeps this list going, right?

:>    There were several issues that I considered when deciding to
:>    create this
:> client. The first was portability. In the end, I decided to make the
:> client in two forms; the first a Java GUI application with optional
:> native methods for select platforms, the second a Java text
:> application with output to the main terminal, again with optional
:> native methods, to be hosted by the user on some unix account, here
:> effectively giving them a way to play the mud as if it were a
:> regular telnet server, with a telnet client. 

:I find the heavy use of native methods intriguing.  Certainly it's
:possible to write a great deal of client code in C/C++ that's highly
:portable.  And let the Java VM handle the OS dependencies.  While it 
:might not give you the highest performance gains in the areas you 
:probably need, it is attractive from a maintenance standpoint.   

This is a big part of it, and something I wouldn't mind doing. It isn't
what I meant, precisely; on the Mac, for example, I have my own version
of the awt that taps into a couple of the newer drawing methods and even
quicktime 2.5 or 3.0, if present. I'm planning to do this for Windows in
time, and eventually possibly for other machines. I haven't learned enough
about BeOS to do this effectively yet, and frankly, don't like the stuff
available with X or Motif, and would prefer to leave that to Java. However...
a lot of the biggest stuff in the client, background graphics display aside,
is text parsing and generation. The neural net code for that is all native,
for good reason; it also happens to be nearly 100% portable. I'm still
working on the portability of the tables associated with it.

:[snip]

:> client on current resources, and I wanted, as a bonus, to be able to
:> provide a GUI and background pictures for clients not running as
:> telnet. 

:Backdrops/backgrounds interest me.  Personally, I would find a client
:quite attractive that included backdrops that represented simple  
:terrain flavors.  For instance forest, desert, meadow or urban 
:backdrops.  Unobtrusive backdrops, like smeared watercolors on light
:canvas that would allow text to be easily read on top.  Sort of a
:mood thing, abstraction is certainly an attractive possibility.  

Watercolors within a short range of the background color (which does change,
and the text color with it for contrast), or, to be more precise, so far
pieces of art based on ... well, check my web page in a week or two, under
the gallery. I'll announce the reopening of my page, as there is going to
be a major mud related section on it. In any case, so far I have a few
hundred pieces of suggestive background, none of which has anything to
indicate specificity, but all designed to suggest a certain mood, clime,
and atmosphere. Day and night, storm and calm, of city, desert, jungle,
forest, town, space, caves, corridors, habitats, suburban towns human and
alien, oceans, islandias, rivers, ice, bunkers, warzones... most of the art
is not production quality yet, but it ought to be VERY effective. I wanted
to use it for Singularity, back when I first tried to tie that old Rom into
a web based plugin server... but it will be a lot more effective with the
dynamic text of Intergalactic War: The Singularity Myth. (The name is a lot
more accurate, and should have been kept as the name for the original. I
have done a lot of going back to original concept with this, and am glad of
that.)

:Most of these images could be included in the client download and my 
:guess(?) is they would be highly compact of their light pixel 
:density.  

I can get an extremely effective 640x480 gif into 6k on average. :) And yes,
an original set would come with the client.

:[snip]

:>    Essentially, the question of client caching comes down to 1) What
:>    are we
:> going to cache, 2) How long are we going to cache it for, 3) where
:> and how, and 4) why do we need to cache it? This brings us to the
:> technical portions of this discourse, and I will now try to break my
:> inane habit of right hand justification. It's become an addiction,
:> you see.

:> 1) What are we going to cache?

:>    There are several potential candidates for caching. Of course,
:>    there are
:> the obvious ones; graphics ought to be cached in memory, then on
:> disk if a disk is available (this is not possible with applets),
:> sounds ought to at the very least be cached on disk; anything else
:> of similar size should, at all costs, be kept as far from repeated
:> downloading as possible. Text, while less bandwidth intensive, might
:> also be a candidate for caching. 

:Fixed or mostly static text like help, documentation, histories, 
:stories would seem to be good candidates, as well as mud mail and 
:bulletin boards. Having the above items client-side would allow the 
:user to utilize mechanisms for off-line reading and study.

There's a good point I hadn't thought of. Well, in a sense I had. Direct
mail doesn't even have to go through the server, if both clients are on. I
have allowed client to client communication in the design. (I have yet to
finish implementing this, and have no idea how well it will work. I based it
on DCC to some degree...) Large amounts of text (the dictionary and
thesaurus) would be built into the client, in rewriteable files that will be
periodicly brought up to date. This might allow easy translation to other
languages. (The grammar is a totally modular component.)

:[snip - security issues]

:>    This covers well the storage of things from the server, but there
:> is another potential here that I think is far more valuable: storage of
:> those things inherent to the client's "memory"... be it name
:> recognition, scripts and preferences, or a complete "personality" to
:> be applied to text parsing or generation, having this on the
:> client's end opens up a whole new level of potential. This is
:> currently embodied by arcade fighting and racing games. Have you
:> ever seen one of these games get tougher and "smarter"?

:Nod.  Lots of potential for client-side programming or preference 
:storage.  In my case, and some others I'd imagine, there would be
:a couple of levels to this user-memory data.  I would make a 
:separation of user memory and character memories since multi-play
:and simultaneous multi-play is a desirable option.

*nod* This is a good point. At the moment, I have several items that are
character specific. Memories of names, a cross-ref for the language files (I
scan the character lexfile first, then the character's jargon lexfiles, then
the global lexfile), preference files (these are models, and not so much a 
character specific thing as a set of preferences that can be applied to more
than one character, loaded or unloaded for a given character, split for some
introduced difference (this is my standard preferences WITH prediction on,
this is my standard preferences WITHOUT...), and so on.

:> 2. How Long are we going to Cache it for?

:>    We've got a chunk of data; we want to be able to reference it
:> without downloading it again; but we don't want to store it forever - unless
:> we are doing the database thing, in which case we may go so far as
:> releasing most of it on a CD and just storing updates 

:I'm thinking in terms of forever for my client-side storage, but 
:understand it's much less graphically ambitious an intensive than 
:your project.  A client-side database updated at will by the server 
:seems to be attractive.  

Physmud isn't that graphicly ambutious. GURU is CD based. Physmud just has
a whopping big load of a lexfile, in general, and with all the crossrefs, it
takes up some large amounts of space.

:> (ref. UOL,
:> M59?), or at the very least, notifying clients that the database may
:> reach a size as large as X, where X is some number larger than what
:> (you hope) is the maximum size in the reasonably foreseeable future.

:Actually Nathan, wouldn't this growth aspect, with respect to 
:graphics-only, be highly predicated on the method of generation 
:and/or the use and types of graphics you send?

Well, yes. And this is a large concern. There are so many ways of producing
graphics. The original GURU was going to be layered sprites with switchable
pallates. As it stands now, I am playing with wireframe and textured surface
models, using caching and stored data...

:If one generates unique pictures, .gif's, .jpg's, anime .gif's etc. 
:(that are likely artist rendered) to represent backdrops, objects, 
:avatars, animation then the longest storage due to frequency of usage
:or appearance might be attractive.  Storage requirements might 
:also depend on the context of the objects' use.  Scenic/room 
:depictions might be cached based on position locality, while 
:creatures might be cached on frequency of appearance.

Again, a good point. At the moment, none of my models distinguish much
between the nature of objects... this presents some problems with the idea
of seperating storage method... but backgrounds alone are easy enough to
deal with. If not cached, they are grabbed in sets (climactic) that tend to
be grouped together.

:If one generates computer drawn images that may or may not depend 
:upon a set of reusable pallettes and textures, one might want 
:permanent storage for any reusable components and some other method 
:for the drawing "instructions" that use them if its ray-tracing, 
:BSPs, etc.   

Such is the format for the GURU project... yeah, I still have to come up with
a good caching method for GURU models and behaviorsets.

:[snip]

:> 3) Where and How?

:>    This is not just a question of memory, disk, or register. This is
:>    also
:> one of access... if we are caching for the convenience of the client
:> and server on downloads, we want it right there, but say there is a
:> memory of some sort associated with the client. Does a player who
:> uses many machines have to carry their memories on a disk? Why not
:> allow a player to set some remote accessable point, in the manner of
:> a remote .newsrc, that can be pointed to by multiple machines, with
:> a breakpoint on time, or a SCCSlike diff function?

:Yikes!  I would have never thought of portable clients in a million
:years.  Do you really think this scenario likely or common?  Or is 
:this one of those administrator-driven features rather than a 
:player-driven feature?  Perhaps a simple method of backing up and 
:restoring the user "memory" area would suffice.  I might be assuming 
:that any graphic data in this area would be of the avatar-type or 
:inventory-type graphic references?  Then again, with the advent of 
:the "internet cafe" you might be on to something with public 
:access terminals becoming as common as phone booths.  :)

I actually think it is common. And yes, I am thinking cafe players, college
students. I am also thinking of personal avatars (for the GURU cached on
the distributed machines, where the closest machine to a recipient actually
drops the copy of the avatar, in the manner of an automated mirror system).
The user memory is NOT graphical for Physmud, at the moment, though I have
designed to allow (DCC face style) personal graphical avatars, and to allow
them to be stored anywhere (http reference protocol instead of that idiotic
DCC upload method, with upload as an option, in other words, first client
either uploads or sends second client an http reference.) These would be
displayed (on any Quicktime 3.0 capable computer... sorry...) in an
activatable window in conjunction with local communication. This also ties
into the direct communication connectivity between clients. The remote 
access files, for physmud, is really just the current states of the personal
preferences, a modification application to the behavioral description file,
and the updates to the memory files, personal lexfiles, etc. Over a certain
number, these modifiers get commited to the upline base version. The entire
set consists of one base version, and five timestamped updates. If your base
on the computer in use is newer than the stored base, you just grab the 
updates since your last version. Someone that only used three computers, for
example (say home, work, and SO's place) would never have to download the big
chunk, just updates. Someone who used a cafe would have to grab the big chunk
every time.

:> 4) why do we need to cache it?

:[no disagreement on the why do's!] 

:I can't understand the need (or desire) for this much backward 
:compatability (back to telnet terminals and 286s).  It's not 
:even reflective of the current mud playerbase.  This is more a 
:reflection of the administrative userbase.

And your point? _I_ want to be able to use this thing myself, you know, and
it isn't that hard to make a UI change in Java, which is all that is being
done here. Think of it like this: how hard is it to write the text output
UI? How hard to write the GUI UI? OK, now, which one is more effective for
a quicky test case? I'm not saying I'm going to use the remote hosted dumb
terminal model much, but... come on, how many of you prefer lynx to Netscape
when just doing a quick check of deja-news? How many prefer vi to Word when
just making a quick edit to a source file? And if I'm going to write it for
myself, why not make it available for others who use telnet access accounts
regularly?
--

Nathan F. Yospe - Aimed High, Crashed Hard, In the Hanger, Back Flying Soon
Jr Software Engineer, Textron Systems Division (On loan to Rocketdyne Tech)
(Temporarily on Hold) Student, University of Hawaii at Manoa, Physics Dept.
yospe#hawaii.edu nyospe#premier.mhpcc.af.mil http://www2.hawaii.edu/~yospe/