[MUD-Dev] ColdStore

[Miroslav Silovic]

J C Lawrence <claw at kanga.nu> writes:

> http://coldstore.linuxbox.com/
> 
> --<cut>--
> ColdStore is a gigabyte-scale persistent object store which mmap()s
> objects into a file (for persistence), can allocate objects in a
> minimal working-set of pages (for speed), permits library versions
> to change without a rebuild of the store, permits application
> checkpoint/restart.  ColdStore might be used to replace a database,
> cache highly structured objects, act as an OODB, make STL
> persistent, act as a Document Store for XML/SGML/HTML, create a
> MUD. Future projects include a virtual machine toolkit, ports of
> several languages, VM and byte-interpretive language workbench.
> --<cut>--

Ok, I did more extensive research into this (mostly because I know the
authors).

It's a C++ library that mmaps a file, and then handles the allocation
and refcounting. This is very fast, of course, as the allocation
handles the locality, but that's the only good thing one can say this
approach. Problems:

 - mmap is limited by the architecture. With normal databases you can
download as needed, and the db file(s) can be bigger than the address
space - if you carefully split your db, for example. With mmap, you're
limited to the largest contiguous segment in your address space, which
really is about 1 gb on most OSes. This seems a lot but I know of at
least one production MUD whose db size hit 1gb at one point (that's
when they rewrote much of their code for data size efficiency) - and
that was Cold db that compresses small integers, so it'd be 1.5-2gb of
pure mmapped binary.

 - bug recovery is nonexistent. If your broken C++ code hits mmaped
memory, you can say goodbye to the entire db.

 - versioning issues. While upgrading ColdStore may be handled, If you
change layout of your objects, well... conversion is impossible, since
you've just mmaped the binary.

 - db format is highly non-portable. It's not just architecture
dependant, it's C++ compiler dependant (as object layout may change).

 - refcounting. Well. Some people swear by it. Usually the same people
who haven't explored its interactions with threading. The issue is
that increasing refcount needs to be done atomically - when you have
LOTS of threads, well... that just won't work. You either have to use
extra spin lock or use inline assembly for atomic increment. And since
you increment/decrement refcounts a LOT (even passing pointers as
parameters may cause refcounts to bump), this is major issue. BTW, has
there been any discussion on the list on incremental/generational GC?
(that works best for MUDs by far, IMHO - you can really bump up your
realtime response once you implement incrementality properly).

 - no way to logically split your db into multiple files. ColdStore
stores direct binary pointers.

I'd say this system is great for small MUDs that need to run on tiny
(read: outdated) servers and aren't changed a lot after the initial
hardcode is in place. It's faster than anything that uses separate db
engine and it handles small dbs efficiently. But I think it'd be a
mistake to use it for a (potentially) large size world.

--
How to eff the ineffable?



_______________________________________________
MUD-Dev maillist  -  MUD-Dev at kanga.nu
http://www.kanga.nu/lists/listinfo/mud-dev