[MUD-Dev] Re: tdbm etc.

[J C Lawrence]

On Sat, 06 May 2000 09:44:21 -0700 
Barry Brachman <brachman at dss.bc.ca> wrote:

> A recent web search turned up this artifact from the past:

That would be:

  http://www.kanga.nu/archives/MUD-Dev-L/1998Q2/msg01081.html

I'm pleased you were able to find that, as its quite an old
message.  Just ouf of curiosity, how did you find it?

>> To: mud-dev#kanga,nu 
>> Subject: [MUD-Dev] Re: Databases: was Re: skill system 
>> From: J C Lawrence <claw#under,engr.sgi.com> 
>> Date: Tue, 16 Jun 1998 11:40:37 -0700
> [...]
>> TDBM is sweet, fast, and rather sexy.  They have some very good
>> discoursory material in the tar ball (or was that the FTP site?)
>> giving performance metrics, design considerations etc.  Very
>> optimised in some ways.  However, for minor unkinked reasons I
>> wanted to move towards a persistent store type model.

> Thanks - I am the author.  I'm pleased to hear you were able to
> make use of the software.

<bow>

I am most grateful for all your hard work.  Thankyou!  A very nice
job you did there BTW.

> I am working on a successor to tdbm and after reading the above
> thought you might be interested.  It's being actively developed
> right now - I'm open to suggestions for features etc.
>
>     http://www.dss.bc.ca/dxstore/

It looks extremely interesting.  I'll have to grab a copy and beat
on it.  As there may be others on the MUD-Dev list interested in
Dxstore, I've crossed this reply to that list.  I trust this is not
a problem.

Quoting some of the feature set from the above URL for MUD-Dev:

--<cut>--
      Immense databases striped over a huge number of file systems
      can be created (RAID level 0). When configured to use 64 bits
      internally, you will almost certainly reach disk space
      capacity limits or operating system limits before exceeding
      Dxstore's capacity; specifically, Dxstore can address over
      2^64 data items, each one of those data items can be on the
      order of 2^64 bytes in size, and the data can be spread over
      2^32 file systems.  This is far greater than the
      commonly-found terabyte limits.

      A Dxstore database looks like any another file on the system,
      so the usual commands can be used to control access to the
      database, rename or copy it, and so on. Any file system that
      supports the handful of system calls that Dxstore requires
      can be used. No special hardware or dedicated disk partitions
      are required and disk space for a database does not need to
      be preallocated. Installation is simple and there is
      practically no database administration.   There's no
      complicated query language to learn or use. A data item is
      stored and accessed using a simple key.   Flexible ways of
      storing and retrieving data are provided. New data can be
      inserted anywhere in a stored value. Any contiguous region of
      a stored value can be retrieved.

      In some databases, the size of a stored value must be known
      at the time the data is given to the database and the data
      must all reside in contiguous memory. With Dxstore, data can
      be streamed into and out of the database; the application
      does not need to tell Dxstore how much data there will be and
      the data does not need to be in memory all at once.

      Sophisticated data editing functionality is provided. A
      region of a stored value can be overwritten or replaced with
      new data. If a stored value represents a table, for example,
      a single row can be replaced.

      Keys can be segregated into keyspaces, allowing small
      databases to be combined into a single physical database.

      Frame memory provides a rock-solid way of preventing memory
      leakage and allows dynamically allocated memory to truly be
      freed and released to the operating system. As a bonus,
      persistent dynamically-allocated memory functionality is
      provided. The frame memory functions are largely orthogonal
      to the database functions; they can be used with little or no
      knowledge of how to use the database.

      Descriptive error codes are made available to the user.

      High performance is paramount. Databases running out of main
      memory, rather than the file system, can be used.   Dxstore
      has a small memory footprint.

      A Dxstore database can be configured to be portable so that
      it can be moved to a platform with a different processor
      architecure.

The design of Dxstore concentrates on providing very fast yet
flexible access to a huge number of data items and vast amounts of
data. Its philosophy generally favours performance over storage
requirements. While Dxstore is not cavalier in its use of disk
storage, disks are cheap and getting cheaper. Time is always
expensive.

No query language is provided, per se; a data item is named by an
arbitrary binary string.  Dxstore is intended to be used by
programmers as a core component of a larger system, acting as the
data storage and retrieval engine, or in applications where support
for complicated queries need not be provided. It can be integrated
into scripting languages, such as Perl, Tcl, and PHP, used by a
full-blown database, object store, or storage area network (SAN)
server.  Dxstore could even be used to implement a user-space file
system.
--<cut>--

--
J C Lawrence                                 Home: claw at kanga.nu
----------(*)                              Other: coder at kanga.nu
--=| A man is as sane as he is dangerous to his environment |=--


_______________________________________________
MUD-Dev mailing list
MUD-Dev at kanga.nu
http://www.kanga.nu/lists/listinfo/mud-dev