[DGD] DGD good choice for MUD with large-scale city simulation?

[bart at wotf.org]

On Sat, 08 Dec 2018 22:35:57 +0100, Felix A. Croes wrote
> bart at wotf.org wrote:
> 
> >[...]
> > > I'm planning a MUD that has a lot of non-standard functionality, 
> > > including an underlying city simulation
> > > 
> > > in which objects (e.g. non-player characters or "mobs") are always 
> > > active, building emergent story lines.
> > > 
> > > Ideally, I'd like millions of active objects. However, due to 
> > > performance constraints I may need to get by
> > > 
> > > with a few thousand.
> >
> > A few thousand should not be an issue. I run DGD instances with a few million
> > objects, many of those being active lots of the time, and this is possible as
> > well, but it requires more care, especially it requires making sure DGD gets
> > to do its housekeeping tasks often enough, and hence ensuring no code ever
> > hogs the system.
> 
> The requirements are quite steep.  Millions of active objects on a
> desktop system?  

Well yes.

> A professional would immediately order a cluster of
> heavy servers for that job.  Then again, it is specifically what 
> Hydra was designed to do, and if the nudlib is designed well, it may 
> just be possible.

Just taking a look at my recently repaired lpcdb test instance, it currently
uses just below 200k objects, most of those can be considered 'active' and a
significant chunk of them gets involved in the queries it processes.

For example: https://fortune.wotf.org/?search=all

This produces just over 3200 results. Any result sent to the browser involves
about 10 objects, and that is just producing the actual result, not finding
it. Obviously it doesn't do all of that in one run, actually it is a chain of
3200+ 'events', each one triggered by the browser responding to having
received the previous result. The actual query doesn't have to involve all
objects as it uses some smart index to do the text search and produces a list
of results, but not the actual results themselves.

This is running on a vm on a desktop system, and after I did some cleaning up
and reworked some nasty bit of code that tried to touch all objects in one
run, it uses anywhere between 0.5 and 1.5gb memory.

Scaling that up to something using over 1m objects requires a fair bit more
memory, and swap, but is within what dgd can handle for this codebase. The
object table is currently limited to 2m objects, and I can hit that limit
before running out of cpu or memory resources.

This is not a trivial system, and took quite a lot of work to be able to
handle this, but... I think you might underestimate how much work can be done
with just a single threaded dgd. I'd assume hydra can be made to handle 10m+
objects with proper care and design.

Bart.
--
https://www.bartsplace.net/
https://wotf.org/
https://www.flickr.com/photos/mrobjective/