[MUD-Dev] Re: TECH: Distributed Muds

[J C Lawrence]

On Thu, 26 Apr 2001 13:39:41 -0400 
Derek Snider <derek at idirect.com> wrote:
> From J C Lawrence:
>> On Wed, 25 Apr 2001 13:02:11 -0400 Derek Snider
>> <derek at idirect.com> wrote:
  
> Here's the scoop: If you page out objects that aren't being used
> currently to disk... guess what's going to happen when someone
> encounters that object? -- LAG!

This is why you do predictive look-ahead for cache stuffing.  Given
a largely idly physical IO system, the latency for phsysical store
requests, while an order of magnitude higher than for in-core data
structures still isn't THAT large.  As long as the rate of incidence
for a given player is not high, given that it essentially happens in
low population cases (edges of the intersection tree) the extra
latency is often easily hidden under the lost noise of the lower
state change rate for that section.  By definition paging can't and
won't happen in the flash crowd population cases -- your total
working set in the crowd case is atypically small there (highly
intersected), and outside of IPC density is arguably the optimal
case in terms of working set operations (great working set
definitions, horrible computational and IO requirements).

> LAG is evil.  Netlag is bad enough.  Server lag is unacceptable.

Server lag when defined in terms of milliseconds for fringe
populations is a near moot point.  We're not talking about
continuous page swapping, but infrequent page swaps.

> It's hard enough eliminating lag from manditory disk operations
> (saving/loading pfiles) without worrying about Virtual Memory.

There's little reason for the other operations, such as pfile saves,
to be synchronous.

> With the cheapness of RAM right now, it's time to turn VM off.  

All of my systems (Kanga.Nu, my desktops, etc) have 512MegB RAM.
They all swap periodically, my desktop especially so.  Maximum
system memory on mid-range hardware (ie reasonably priced in
MIP/$$$) at this point is in the single digit GigB range.  I don't
know the current total data set size of services like UO, EQ, etc,
but I would be unsurprised if they are in middle double digit GigB
range with future larger services heading for the ~100GigB range.

  Consider a single-image MUD with a 10^6 active simultaneous player
  count.  Now add the supporting data sets needed for a world large
  enough to support that sort of active player population *AND* to
  support the inactive player population (ie those who will logon
  later to replace the currently playing population).  100GB starts
  looking quite possible.

The current solution to the data set size problem is to fract the
data set onto discrete hardware systems such that the total dataset
for a given node can be represented in core memory.  (Typically this
is done along zone lines) I don't see this as a tenable model over
even the mid term, let alone the long term.  

> VM is fine for desktop machines, not for servers.  Those days are
> long past.  Once your server starts swapping, performance goes
> down the toilet.

This depends on what the swap rate is.  A page swap rate of 1/minute
is quite tolerable.  A page swap rate of 1K/second is not.

> You say you have 10Gigs of game world data?  Looks like you've got
> to spend $3000 on RAM spread over a few zone servers.  Big deal.

What if zone segmentation doesn't work anymore?  

We are entering the realm of actually Hard Problems.

> When you have a UNIX server paging out idle memory pages, the only
> way to keep pages in memory is to access them regularily, and the
> only way to do that is to have a regular periodic traversal of all
> objects in memory.

Or you lock them.

Or you redefine your VM's idle definition (several commercial Unixes
allow this to a minor extent).

Or you reconfigure your VM system's page cache algorithms (several
commercial Unixes allow this).

Or you build your VM so that it cooperates predictively with the
process scheduler and physical IO subsystem for predictive latency
minimisation (I'm only aware of IRIX and AIX doing this to any
interesting extent).

--
J C Lawrence                                       claw at kanga.nu
---------(*)                          http://www.kanga.nu/~claw/
--=| A man is as sane as he is dangerous to his environment |=--
_______________________________________________
MUD-Dev mailing list
MUD-Dev at kanga.nu
https://www.kanga.nu/lists/listinfo/mud-dev