[MUD-Dev] TECH: DBM vs BDB speeds (was: AmigaMud DB questions)

[Jon Lambert]

Bruce wrote:
> Jon Lambert wrote:

> I would love to hear that I'm wrong or that I missed something and
> that I can get equivalent levels of performance out of GDBM. :) It'd
> make life much easier on me to not have to introduce a dependency on
> BDB 3.2.9 for Cold, or alternatively, to have to maintain 2 versions
> of our lookup index code.

The only work I know of that has looked at optimizing massive
insertions in particular has been TDBM and dxStore.  I believe similar
problems prompted the creation of both.  Neither is quite as mature as
GDBM or BDB (meaning possibly buggy).

> Cold does store 2 records per object.  But the first, symbol->objnum
> is cached within the driver which avoids most of the common hits to
> that DB.  Currently, objnum->seekaddr isn't cached (I'm not sure
> why, this is old code), but I've recently introduced some code to
> avoid re-writing that value if it isn't necessary to do so.  (That
> change has no impact o the DB compilation phase which has been our
> main worry.)  Currently, our disk I/O patterns only involve the
> objnum->seekaddr index and the block file.  With BDB and an
> appropriate cache size, we should be avoiding most of the index
> updating.

I created a special object called $catalog.  It's sole function is
maintaining those global symbols.  And by globals I mean names of
objects.  Anyways since its just another object it lives in cache with
the rest of objects.  And the high access frequency tends to keep it
there.

I suppose it might be a useful concept for Cold as it would
necessarily remove any objname->objnum index records, and make the in
memory hashtable unecessary.  Of course in your particular case, that
would probably be one massive object.  :-)

> Since Cold requires a decompile/recompile of the binary database
> when doing upgrades of the driver, this was very important work as
> it dropped the required maintenance window for upgrading a large
> game from somewhere over 15 hours to under 3, possibly under 2.
> (For a typical small game, you'd be seeing decompile/recompile times
> that are well under 10 minutes, both with the old and new lookup
> index code.)  This work also resulted in us returning to being CPU
> bound during DB compilation.

I guess maybe if the problems of why it's necessary to do a decompile
and recompile are resolved when one does a driver upgrade then it
wouldn't be that big of an issue.  I assume that data changes in some
dramatic way between driver updates?  It shouldn't.  Which is to say
that I don't think it should at this late stage in development.  And
if it did some sort of automatic upgrading would be more useful.

--
--* Jon A. Lambert - TychoMUD        Email:jlsysinc at ix.netcom.com *--
--* Mud Server Developer's Page <http://tychomud.home.netcom.com> *--
--* If I had known it was harmless, I would have killed it myself.*--
 
_______________________________________________
MUD-Dev mailing list
MUD-Dev at kanga.nu
https://www.kanga.nu/lists/listinfo/mud-dev