[MUD-Dev] ColdStore. Belated response from a developer.

[Miroslav Silovic]

colin at field.medicine.adelaide.edu.au writes:

> ColdStore compresses small integers too.

Could you post a bit more about ColdStore internals on this list?

I was under impression that it used plain virtual memory to talk to
hardware (thus getting speed).

> I'd have thought the storage between Cold and ColdStore was
> comparable.

The last version of Cold storage is my work, and one I'm not
particularily proud of (okay, this is an understatement. In actuality,
it's a brown paper bag quality code - I want to wear one over my head
when I think about certain things in there). :)

> ColdStore doesn't set out to make C++ programming safe, because
> that's impossible.  It sets out to provide a toolkit for supporting
> higher-level interpreted languages in which one would write
> applications.

The website suggested it was C++ persistent store system, I was
replying under that assumption.

> It's certainly necessary to thoroughly engineer the C++ classes you
> use and to avoid world-stopping bugs, however Miro's argument could
> apply to any complex piece of code upon which your world depends,
> the o/s, the db backend.

'Don't write buggy code' is not something I'd recommend to a
programmer and expect gratitude. :)

> You could look at ColdStore as a C++-extensible database engine
> which has been optimised for the kinds of storage one would require
> for MUDs.  Certainly, if the database is buggy your MUD will be
> toast.  Recommend: don't extend the db with buggy code.

Ditto above.

> One could certainly use the toconstructor() facility for changing
> the raw layout of objects.
> 
> This would be painful, however, and one would be well advised to
> think through the raw layout of lowest level objects before
> committing them to persistent store.

Assuming that ColdStore is not meant to store C++ objects, but only be
used with an interpreter to keep it in check, this makes sense.

> >- db format is highly non-portable. It's not just architecture dependant, 
> it's C++ compiler dependant (as object layout may change).
> 
> Certainly.  It was a design decision to localise the cost of moving
> between architectures to the time when one might wish to actually
> move them, rather than seek to support an abstracted/portable format
> for which one pays on each access.
> 
> Many database formats are not immediately/binary portable, usually the 
> databases have to be serialised or marshalled before moving them.

Does ColdStore support this, or is it supposed to be provided by an
application sitting on top of it?

> I was in the process of writing it right out in favour of GC, when
> someone pointed out to me that refcounting is a great storage
> optimiser - ColdStore collections and vector types share all
> substructure, and implement Copy On Write semantics on low level
> objects.

This is acceptable if you already decided to pay the penalty for
having an interpreter in the first place.

> I don't see the problem with using inline assembly for atomic
> increments, that's what inline assembly is for.  It's what we use,
> nicely wrapped in a Counter class.

*.......* (insert any number of sounds here)

> Maximising data locality by sharing substructure will always tend to
> increase contention between threads at the same time as it tends to
> decrease page faults.  This is a classic trade-off: throughput
> vs. responsiveness.  Are networks really fast enough, yet, that
> response-lag is an issue?
> 
> As long as the time taken to swap a page from disk is much longer
> than a thread context switch, I'll tend to worry more about locality
> than thread performance, most particularly since most of large MUD
> architectures don't even have real multitasking.

Okay, I think this is the central issue.

Yes, textual MUDs are bandwidth-bound. However, this list is not only
about textual MUDs. Also, I think that bandwidth-bound MUDs are not
interesting any more (because even MOO, once that memory became cheap,
CPUs became fast, and codebase became slightly more optimised than it
used to be, is now bandwidth-bound - so why bother inventing a better
mousetrap?)

Now, if you have a predictive client (one that keeps some state, and
knows how to evolve and display it), you can just send diffs from the
server, saving LOTS of bandwidth. In that case, the interesting things
become possible, but the server bottleneck shifts from bandwidth to
processing. Refer to this list's archive for examples.

Notice about paging - you have to write to the page each time you
increase the refcount, and refcount may get increased just by passing
pointers around. This is different from GC schemes, where you only
touch the pages you really want to write to (the tradeoff being the
need for write barrier, with any decent GC). Then there is robustness
of GC compared to refcounting (as reported by people involved with
Mozilla project). Refcounting certainly makes sense if the ammount of
the code in native language (C++, in this case) is small - but I don't
like assuming that for a low level system I'd like to see in wide use.

> Additionally, the authors of Texas don't answer email, the build
> process entails heavy use of a hacked gdb (using STABS to generate
> object interpretation code for swizzling) and Texas has no support
> for dynamically loading new objects into a store.

I actually meant to say that Texas is a cool concept whose
implementation desperately needs a complete rewrite, preferably by a
team of programmers (rather than a team of computer scientists). :)

To conclude, within the assumptions you decided to work with,
ColdStore seems cool (modulo my disagreement with use of inline
assembly). But I tend to work under VERY different assumptions. :)

--
How to eff the ineffable?



_______________________________________________
MUD-Dev mailing list
MUD-Dev at kanga.nu
http://www.kanga.nu/lists/listinfo/mud-dev