[MUD-Dev] Re: TECH: Distributed Muds

[Bruce]

Vincent Archer wrote:

> A friend looked at what we were trying to do and said: "it's a
> trap".  Because you no longer know that objects are now in fact
> stored on disk, and not resident in a process memory, you quickly
> fall into traps. Like searching for an object by scanning a linked
> list. Ahhh, but the linked list will quickly pull 1Gbyte of data in
> your process just to find the object that has this property or
> that. And worse: you're not conceptually aware of it. Until you try
> the program on the real data, and finds out everything crawling at
> speeds that would make an old 8086 seem a decent system :)

Sure.  And the same is true with any number of things, like using more
memory than you have available and causing your system to thrash.
But, in the end, that's something that you're constantly aware of when
programming and something that you can monitor easily.  Depending on
your tools and such, you can even track some of these problems down
fairly easily. (Mozilla's tracemalloc is pretty useful for that type
of thing.)

The same with an OODB.  Fundamentally, an OODB is simply a way to help
you manage situations where your dataset's size is in excess of the
amount of available memory.  In one situation, with an OODB, you can
establish cache sizes and attempt to intelligently manage your working
set.  In the other, you can just read in data and allow the OS to
manage your working set for you.  You can just get used to watching
your working set of objects, just as you watch your working set in
memory. With Cold, I plan to go a step further in helping the
programmer monitor this.  Soon, I'll be implementing some basic
accounting within the driver to track statistics about tasks.  While
this is something that is long overdue, it'll hopefully provide us
with the ability to detect and track tasks that are taking longer
periods of time as well as accessing more objects either within the
cache or on disk.  This'll be a nice complement to the existing set of
tools that we have for monitoring DB accesses and DB information
within Cold and should address the types of concerns that you raised.
When letting the OS manage your working set, this type of information
is significantly more difficult to get to the same levels of detail
and usefulness.

I think that the issue of the use of an OODB is different from what I
was questioning with DSM or RMI type systems.  In those, I don't
-know- of tools to help debug problems or even notice them.
Additionally, with the matter of an OODB, you can assume "oh, I'm
accessing another object, do I want to do that?" whereas the type of
thing that you're doing with a DSM or RMI operation is something that
is meant to be indistinguishable from a cheaper operation, that of
accessing a value in memory or performing a method call.

Once the design has gotten out of control, how easy is it to fix
things? I honestly don't know when the subject is working with a DSM
system or with RMI, but I'd be really interested in knowing about
people's experiences here.

  - Bruce

_______________________________________________
MUD-Dev mailing list
MUD-Dev at kanga.nu
https://www.kanga.nu/lists/listinfo/mud-dev