[MUD-Dev] [Tech] MUDs, MORPGs, and Object Persistence

[brian price]

> From: Daniel.Harman at barclayscapital.com
> To: mud-dev at kanga.nu
> Subject: RE: [MUD-Dev] [Tech] MUDs, MORPGs, and Object Persistence
> Date: Fri, 11 May 2001 13:35:16 +0100
> Reply-To: mud-dev at kanga.nu

> -----Original Message-----
> From: brian price [mailto:brianleeprice at hotmail.com]
> Sent: 11 May 2001 00:27
> To: mud-dev at kanga.nu
> Subject: [MUD-Dev] [Tech] MUDs, MORPGs, and Object Persistence

>> For purposes of fault tolerance, we need a datastore that can
>> periodically be backed up in a fast and efficient manner,
>> preferably without stalling the server.  Note that transaction
>> capability (in db terms) is *not* a requirement, the capability of
>> generating checkpoints by writing out the 'dirtied' (changed)
>> portions of the database periodically will satisfy the datastore
>> backup requirement.  Checkpoints can be restored simply by starting
>> with the last full backup and applying (in order) the saved changes
>> since that backup occurred (can be done offline).

> I disagree with you on a lot of points here, but I'd start here. I
> think transactions are important in a MUD. Its the best way to
> prevent duplicates through synchronisation problems (which is how
> most of the EQ ones I heard about worked). If someone giving an item
> to someone else can be made with a call to a single transactional
> 'switchObjOwnership()' type method, then you aren't going to get
> either dupes or item loss when passing items.

You can only get dupes or item loss in a system that does not capture
coherent snapshots of the persistence state during safe periods.
Consider: A hands item to B; both objects A and B (and in some cases
the item object itself) are dirtied (persistent state changed).  If
you create a checkpoint prior to the exchange, any rollback results in
the restoration of the state prior to the exchange.  If you create a
checkpoint after the exchange, any rollback results in the restoration
of the state after the exchange occurred. You only run into problems
when you try creating a checkpoint in the midst of an exchange -
simply designing the server system so this condition cannot occur
solves the problem.


>>  Thus our datastore requirements are:

>>   1) frequent writes of dirty objects
>>   2) infrequent reads of collections of objects
>>   3) large number of classes
>>   4) deep inheritance trees
>>   5) fast and efficient backups thru use of checkpoints or equivalent

>>  An OTS or OS RDBMS is not overkill given these requirements, in
>>  fact, the entire class of available RDBMS solutions are
>>  underpowered, inefficient, slow, and expensive.  An oft touted
>>  feature of most RDBMS - SQL - is, in this case, completely
>>  unnecessary *and* imposes a significant performance hit for zero
>>  gain.  Worse, the use of efficient objects and resultant class
>>  bloat is practically impossible to represent in RDB terms without
>>  investing an insane amount of development time.

> You previously said that infrequent reads were required. Thus I
> don't see how the performance of an RDBMS is going to impact your
> proposed solution.  Writes are generally fairly fast, its the
> queries that are slow.

Writes are going to be slower in an RDBMS than a OODB/Persistent
Object Store due to the large number of indices and tables typically
required in an RDB model that is used to represent an object model.

> By not going for an RDBMS you have made any type of reporting
> functionality many times more difficult to implement. If you have a
> large game, then I would imagine functionality to measure how many
> warriors have weapons of greater than 'x' affectiveness is something
> you might want to find out infrequently enough to make writing a
> bespoke tool a pain, but frequently enough that having sql is a
> feature. The same for economy reports and such like. With a bespoke
> object store, any type of data-mining is just hideous.

In previous discussions elsewhere, the maintenance issue has been
raised time and time again as a reason for choosing RDBMS over
OODB/Persistent object store.  There is an alternative for OODB that
is in many ways far more powerful: embedded script/interpreted
language engines.  For C++ server implementations, Java or Python are
natural choices.

> Anyway, a well tuned and designed database can be remarkably fast.

Performance issues boil down to the number of file i/o operations
required per equivalent action.  In the case considered in the
original message, the OODB/Persistent object store approach results in
far fewer file i/o operations than the equivalent RDBMS solution for
the actions required.

>>  IMO, the best solution is a persistent object store.  Not a full
>>  fledged OODB (whatever that is), but a collection based storage
>>  and retrieval system for serializable objects.  In C++, such a
>>  system is fairly easy to develop: combine a RTTI based object
>>  persistence layer with the idea of 'data objects' using
>>  proxy/accessor pattern (to hide object memory presence and control
>>  object memory lifetime) with an object cache and a simple db store
>>  consisting of one index (object ids) and one table with records of
>>  the form: <object id> , <class id> , <serialized object data> .

> RTTI is plain slow and seldom justified.

RTTI is used in my persistent data stores simply as a fully C++
compatible method of associating ascii class names with class
factories.  Since dynamic casting is not used, the RTTI performance
penalties are not a factor.

> If you work with an interface/implementation design pattern, then
> its better to have a persistence interface imho.

The system I have based the original message on uses a persistence
interface defined as an abstract base class with pure virtual methods
for load and store.

> Personally I'd have a couple of methods that could stream and
> unstream an object (for transport between servers - after all we are
> talking distributed large scale muds here right?),

Actually I was not speaking of distributed server implementations in
the original message (sorry for not making that clear).  I've been
considering the case of a MUD/MORPG running on a single machine such
as a quad 700+ Mhz PIII with 2GB RAM.  However, I believe the approach
might be extensible to such an implementation although I have yet to
thoroughly examine the case.

> and a persist method to get it to write itself to the db. None of
> these are a great deal of work. If you were to go towards Java or
> C#, you could make this even more trivial with the object
> reflection.

It seems you're speaking here of translating to/from the rdb model in
each object's persistence interface.  For simple systems this would
work fine, but with a large number of classes I'd think both
implementation and maintenance would become a task of herculean
proportions.

>>  I've heard all the arguments against OODBMS over the years and all
>>  the arguments for RDBMS, and in this case at least, *none* of them
>>  hold any water.

> I disagree. I think an RDBMS with a bespoke in-memory cache would be
> the optimal solution.

The very need for a seperate cache makes such a solution non-optimal
in the stated case.

> What about failover? A proper RDBMS will faciliate this. I get ill
> thinking about having to write one of these for some kind of bespoke
> flat file object store.

Failover is a non-issue because the persistent store is tightly
integrated with and local to the MUD/MORPG server.  Even in the
distributed case, depending upon system design, it may not be
necessary.

> Its interesting, because I have worked on two version of a
> large(ish) scale distributed fat-client system, one where we used
> sysbase, and another where we did use a bespoke flat file system
> with in memory cache for 'performance' reasons. The flat file system
> whilst initally fast, was in fact more trouble than it was worth for
> the following reasons:

I do not believe the application spheres are congruent.  We can
compare apples and oranges all day in re RDB/ODB.

Brian Price
-= have compiler, will travel =-
_______________________________________________
MUD-Dev mailing list
MUD-Dev at kanga.nu
https://www.kanga.nu/lists/listinfo/mud-dev