[DGD] DGD/MP & OpenMosix revisited?

[Greg Lewis]

On Mon, Feb 09, 2004 at 08:54:23PM +0100, Felix A. Croes wrote:
> Greg Lewis <glewis at eyesbeyond.com> wrote:
> > > Then there is the software maintenance cost.  All of these machines
> > > have an operating system which must be installed and kept up to date.
> > > Any OS problem is potentially multiplied by the number of machines.
> > > The hardware could perhaps be managed by a single person, the software
> > > can not.
> >
> > Sorry, but I have to disagree with this entirely.  In a cluster it is
> > customary to manage the operating system for the nodes as images.  In
> > MUD cluster I'd expect there to be two production images, one for the
> > standard compute nodes which are running the game and one for some
> > I/O nodes which are managing state dumps/database transactions.  You
> > do _not_ manage the OS for each individual node unless you're a masochist.
> >
> > State of the art is to clone these images onto the relevant nodes over
> > a management network (avoiding congestion on the main interconnects)
> > using multicast.  The nodes themselves have network bootable NICs using
> > PXE or have something like etherboot flashed onto the NIC ROMs.
> 
> That's installation and and upgrading taken care of.  I have my
> doubts about other maintenance, but I'll just cede the point.

There are a number of solutions here:

1. If you have to, something like pdsh (a parallel shell) could be used,
   however this gets ugly with a lot of nodes.  There are similar tools
   which allow the copying of files or execution of commands to happen on
   multiple nodes at once.
2. Perform your maintenance on a single node.  Use that node to either
   create or update an image and simply reimage the other nodes.

The goal is, of course, to just manage and maintain the images, not the
node.  Things are getting pretty close in this area.

> > > Add to this the cost of designing your MUD to be distributed (which is
> > > not trivial), the cost to the MUD itself of being distributed (some
> > > features that you'd really like to have are just not going to be
> > > possible), the cost of making sure that the MUD software keeps working
> > > as intended in this distributed environment, the cost of synchronizing
> > > MUD software updates across all machines, and probably quite a few more
> > > things that I didn't think of -- at this point, you are paying a lot of
> > > people to manage your overhead of cheap hardware on a fast network.
> >
> > Designing a distributed MUD is certainly a challenge, at least we agree
> > on this one :).  However, this is where a transparent process migration
> > technology like OpenMosix of bproc can help simplify things.
> 
> This point I won't cede.  Some things just don't scale well using
> clusters, and in my opinion that includes MUDs.

I think thats the interesting question here :).  How hard is it to design
a MUD which will run in a clustered environment?  I'm interested that you
see this as quite a lot harder than designing a MUD for an MP system.
Maybe you could elaborate on that?

> I hope to take this out of the realm of argument, and into the
> realm of demonstration fairly soon.  Should DGD/MP turn out to be
> a miserable failure, feel free to call me a fool. :)

My apologies if I came across as overly argumentative.  I doubt DGD/MP
will be a failure :).

-- 
Greg Lewis                          Email   : glewis at eyesbeyond.com
Eyes Beyond                         Web     : http://www.eyesbeyond.com
Information Technology              FreeBSD : glewis at FreeBSD.org

_________________________________________________________________
List config page:  http://list.imaginary.com/mailman/listinfo/dgd