[MUD-Dev] ColdStore

[Miroslav Silovic]

Cynbe ru Taren <cynbe at muq.org> writes:

> I'm particularly interested because I'm planning on converting Muq
> to using mmap() next year.  Your comments suggest that perhaps I
> should retain the ability to run non-mmap()ed when I do.

Note that mmapped is okay if you make it possible to serialize the
object in some intelligent way and then copy them into the mmapped
memory.

> Did you mean "most 32-bit OSes"?

Of course. Thought I mentioned 32-bits somewhere above but on
rereading the post, aparently I didn't.

> I don't mean to carp, but outside of Intel x86 boxes, 32-bits seems
> basically dead, and I'd presumed that the 64-bit world was roomier
> than this?

Hmm. 64-bit is available on Alphas, and on SUN Ultras if you install
Ultrapenguin (I was told that to run 64-bit code on Solaris, you need
to open the machine and switch a jumper - aparently it's some sort of
a bug workaround. They may have fixed it by now, though). HPUX and AIX
are both 32-bit and will stay that way till Itanium begins shipping
(that's in a year).

IMHO you will almost certainly run your MUD on I-32 for the next 2
years. So, I stay with my 'most OSes' claim. :)

> You obviously have alternatives in mind, but they aren't clear to
> me.

Cache + separate db is one alternative - this involves one more layer
of copying, in exchange for safety (and other things I mentioned).

You also have Texas Persistent Store (Altavista should be able to tell
you about it). It works by making pointers to unloaded objects point
to unreadable/unwritable page.  This causes segfault on object access,
and in the segfault handler you can munge the object contents without
problems - namely, you demingle data pointers from object IDs to real
object pointers and you track all the pointers to the original page
and make them point to the real object instance.

ColdStore authors chose not to implement this because, according to
them, it performs really slowly.

> I'd guess broken C++ code hitting memory could in principle trash
> any db, but you see the risks as several orders of magnitude greater
> when the entire db is memory-mapped, vs just a cache of it?

Yes - when you bomb your cache and segfault, it's likely that
serialisation code will bomb as it runs into inconsistent data, or
that the server will break before it manages to write out the faulty
objects. In both cases, inter-object consistency will break
(i.e. objects may dissapear from their location's content
list). However, if you only write out your cache periodically (like
every 2-3 minutes) or use transactional database (see archives), this
won't be an issue, either, and your server is free to crash at will
*grin*

> Or do you have solutions in mind for doing bug recovery well in
> mmap()ed implementations, which ColdStore merely isn't implementing?

No, apart from mmapped dbm with API layer. Note that a program is far
less likely to wildly trash the memory than to overwrite the contents
'near' the objects currently being manipulated (that's what index
overflows, writes into unmalloced, and writes into freed
do... although if you manage to overwrite data structures used by
allocator, you're out of luck).

Things change when you ditch C++ altogether, though - there is a
number of languages that don't allow you pointer arithmetics. This
seems sucky because you can't implement array datastructure - but
these languages generally come with arrays builtin. However, once you
ditch pointer arithmetics, you also can't implement fancy allocation
schemes by hand, so these languages also come with garbage collector
built in. Java is a prime example (and with gjc, you can compile it
into native code). Others are Sather, TOM, ML, Scheme (with Stalin and
Hobbit compilers), Common LISP (with CMU CL compiler). Needless to
say, none of these will allow for mmaps, either. Tradeoff is that you
never again have to worry about memory bugs (modulo compiler bugs).

> Is the db format COFF?  Or what was the relationship to COFF?  Or
> am I misunderstanding completely some of the ColdStore-related
> traffic I vaguely remember?

They intern ELF symbols to keep updated with code changes (i.e. with
moving method implementations). The problem is in data layout within
the objects. I.e. adding new properties. And in the fact that C++
compilers may reorder properties at will.

Let's be more specific about ELF interning (quoting from ColdStore
docs):

------------
I realise that this decision entails considerable portability
problems, and probably binds me to the distribution of special purpose
link loaders, but I think it's the right technical choice, and I think
the functionality should probably be present in standard libdl.nso.

Since I can't even get an unequivocal consensus from the glibc team
that relocating absolute symbols is a bug as well as a contradiction
in terms, or that it doesn't conform to ELF standards, I hold no hope
of having useful (but fairly specialised) functionality added to
libdl.so unless I do it myself.
------------

>From what I gather in their docs, this system is Linux/gcc-only.

> You sound knowledgable:  I'd be interested to hear you comment further.

I'll post it in a separate thread. :)

--
How to eff the ineffable?



_______________________________________________
MUD-Dev maillist  -  MUD-Dev at kanga.nu
http://www.kanga.nu/lists/listinfo/mud-dev