[MUD-Dev] Re: DIS: Client-Server vs Peer-to-Peer

[Greg Underwood]

At 06:56 PM 11/30/98 +0100, Niklas Elmqvist wrote:
>On Sun, 29 Nov 1998, Greg Underwood wrote:

<...>

>> I've been working with those DoD DIS networks you mentioned for about 2
>> years now, so I thought I'd chip in my $0.02.  You may want to look into
>> HLA, instead of DIS, btw.  DoD is cutting all funding to all DoD DIS
>> projects, and shifting it over to HLA.  However, as I understand it, HLA=
 is
>> essentially DIS++ (DIS plus a couple extra protocols).
>
>Oh good! Then you can be my very own DIS consultant in this project :)=20

Ok, at standard consulting fees of $60/hr (US) that'd be...=20

;)

>As for HLA, yes, I've read about it, and I actually included it into my
>project proposal. If you could provide a short summary, I would be much
>pleased. :)=20

I'll have to look into it some more.  All I really know about HLA is that
it's DIS++.  However, since I am one of those DoD guys who may be affected
by that mandate, I have a feeling I will be learning a lot more about HLA
in the near future.  I'll post a summary once I learn enough to sumarize. =
 :)

>> At 10:06 PM 11/24/98 +0100, Niklas Elmqvist wrote:
>>=20
>> <...>
>>=20
>> >Here's the situation:
>> >I'm currently drafting a proposal for a group project course me and my
>> >fellow classmates will undertake next year. This course will include a
>> >range of several different projects which all students will be able to
>> >choose from -- I am aiming to contribute one of the project ideas.
>>=20
>> I'd be interested in seeing the proposal posted to this list, once it's
>> done.=20
>
>It IS done (I had to scramble, since the deadline was close), but I doubt
>it is much good to most people around here -- it's in Swedish. However, I
>will probably translate it (it looks good to have an English version as
>well) and post it to this list for feedback. In addition, I am planning to
>put up a little DIS resource on Artoo (my webserver), where I will put a
>summary of the MUD-Dev responses to my post as well as some other goodies,
>design documents and resources. If the project is accepted (and if I am
>allowed on the team), this site will grow considerably.=20

Spiffy, spiffy and more spiffy.  :)  I look forward to the english
translation.  About the closest I come to Swedish is 4 years of German in
HS, and a refresher course in College.
=20
>In other words, if you are interested, watch this space :)=20

Will do!

>> I'd be surprised if you need to deviate from the standard any.  It is=
 VERY
>> broad, and any *fully* DIS compliant product should be maliable enough=
 that
>> you can do a lot of what I think you want to do for this project.
>
>Really? I thought the peer-to-peer architecture permeated the standard? I
>read a DIS summary somewhere (I think it was on M=E4k Technologies=
 homepage,
><URL: http://www.mak.com>), and they listed peer-to-peer, distribution as
>in targeted clients deciding on damage, dead reckoning, and so on as the
>key characteristics of DIS.

I re-read my copy of the standard.  I believe the idea of PtP was foremost
in the mids of the designers of the standard, but I don't think the wording
of the standard prohibits a carefully constructed C/S implementation.
Nowhere does it say that all applications within the exercize must be full
blown peers.  In fact, in the latest copy of the standard (1278-1a), they
discuss applications with differing levels of granularity/fidelity.

A C/S implementation is bending away from the spirit of DIS, but I don't
think it breaks the standard in any way, and nowhere does the standard
explicitly state that it must be a PtP architecture.

>> >One of
>> >the keypoints of DIS is that it is a strict peer-to-peer architecture,
>> >which means that there is no central server which makes the decisions=
 and
>> >calls the shots.
>>=20
>> Yes and no.  There are simulation control PDUs which can be sent (stop,
>> start, destroy entity, create entity, etc), and I believe there is a
>> concept of a simulation controller within the standard (I'd have to go=
 into
>> work to get my copy). =20
>
>Hmm. Are you aware of somewhere I can get a good overview and reference of
>the standard (preferably in PostScript or PDF format suitable for
>printing)? IEEE seems determined to keep their standards off-line (I am
>told that selling standards is their main source of income).

I couldn't find a web version.  I had to resort to purchasing it from the
IEEE press.  I don't know about their finances, but it wouldn't surprise me
if your statement was false.  One of these days I'm gonna have to break
down and join them... :)

>> You should be able to make a Client-Server architecture out of a DIS
>> system.  It is mostly implemented as peer-to-peer, but is not so by
>> design of the standard.=20
>
>Ahh, good. Standards are standards, and should be followed. (Hopefully,
>there is enough leeway in it to allow us to do our own design and analysis
>-- I'd hate to just steal someone's canned system architecture.) Indeed,
>it feels a little more reassuring to have a standard to fall back on
>instead of striking out on our own completely without precedent.=20

*nod*  Be aware that a C/S architecutre is, as I stated above, bending the
spirit of the standard a bit, but by no means breaks it.

>> > For example, in the DIS standard, the shooting client is
>> >responsible for reporting the shot and the ballistics to the targeted
>> >client, but it us up to the targeted client to decide whether the shot=
 is
>> >a hit or not, and the extents of the damages.=20
>>=20
>> Well, techinically, that is what the standard IMPLIES, not what it=
 states.
>
>Really? That's what I read on one of the summary pages, and that was what
>I seemed to recall from reading the standard a while back.

The standard states (paraphrased):

  Upon receipt of a Detonation PDU the application controlling the entity
targed  shall determine the effects, if any, of the detonation and report
any changes in the entity's appearance via an Entity State PDU.

Nowhere does it say that the application controlling the entity (the one
that resolves the detonation result) has to be the application that
provides the user interface for a man-in-the-loop sim.  In fact, the
standard says absolutely nothing about UI for MITL sims.

>> All it states is that the Detonation PDU has to be issued from the
>> simulation that detects the detonation (note that this does not have to
>> be the simulation controlling the entity that detonated).=20
>
>Aha! So this means a server could be made responsible for this, not the=20
>clients?=20

yup.  The server could detect the detonation and send out a PDU to all the
clients, so that they can show an explosion, make a loud noise, tip the guy
in the rumble seat out of his chair, etc.  The only application that does
more than that is the one "controlling" the entity, which, as I said above,
does not have to be the one the UI is implemented in.

>> It is a common (and, in a trusted network, safe and easy to implement)
>> assumption that the simulation controlling the targeted entity resolves
>> the damage.  However, if you wanted to have a Client-Server
>> architecture, you could have the simulation controller resolve all
>> damage and send out sim-control PDUs to enact the proper results.=20

Note that my above statements about "controlling" the entity do contradict
this paragraph.  I should have read my copy of the standard before jumping
in.  :)

<...>
=20
>> The clients are stripped down to simply provide the interface to the
>> environment, managing their own entities, and dead-reckoning the foreign
>> entities, between state updates. =20
>
>Let me get this straight -- in this setup, the clients only get notice of
>the foreign entities that it can see?

Not necessarily, though that would be a good means of reducing network
traffic, and adding another layer of security to the system.  The server
could selectively send PDUs to clients, to reduce the number of PDUs being
sent, and to reduce the amount of extraneous info being sent to clients.
It would be quite easy for a user to modify their client to show them where
everyone is hiding if all PDUs were sent to all clients.  :)

>> If the client detects a collision/detonation, it fires off the
>> appropriate PDU. =20
>
>But the server keeps track of collisions/detonations as well?

You could do that.  In an untrusted network, you had best.  How much trust
you put in your clients is up to you.

>> The Server catches any detonation/collision PDUs, and immediately sends
>> out an entity-Stop/Pause PDU. =20
>
>This is the standard way of resolving this in a DIS? The affected clients
>are suspended until the server gets around to resolve the
>detonation/collision?

No, because most DIS apps are PtP on trusted networks.  Damage resolution
occurs in the app controlling the entity.  If you want to step away from
that design, and go C/S, then you may find it necessary to suspend the
client, temporarily, until the damage can be resolved.  I should have
wrapped that part with a <thinking as I type> </thinking as I type> tag.  ;)

>> Once it resolves the collision/detonation, it sends out a PDU (can't
>> recall the name, I think it's an Entity State PDU, but it might be a
>> special PDU) to update the targeted entity, and then a Start/Resume PDU.=
=20
>> If it decides the entity is dead, it sends out an Entity Destroy PDU,
>> telling everyone that the entity is dead. =20
>
>Everyone? Would this be required in this setup using a C/S-architecture?=20
>Seems to me only affected clients (i.e. clients in range of the dead
>entity) need to know this.=20
>
>> The clients would have to be set up to ignore all PDUs from any entity
>> they didn't receive a Start PDU for (to prevent hacking of the client).=
=20
>> The problem with this is increased latency between action and
>> resolution.=20
>
>Hmm, not sure I fully understand the implications of this. Are clients
>still exchanging PDUs between each other in a C/S setting? I thought all
>PDUs should be routed through the server -- that is, that all
>communication is carried out between client and server, and not between
>clients.

Hrm.  Good point.  <bow>
The only sticky point I can think of is that DIS does lay out the means by
which PDUs have to be transmitted betwixt simulations.  There are several
options, but they might all force something more PtP.  I think I need to go
read the first section of the standard again (left it at work again..
grumble).


>> > - Is it possible in an Open Source project where anyone has access to=
 the
>> >	source?=20
>>=20
>> Only as a Client-Server model.  And as long as you, and/or trusted
>> flunkies, are the only ones with access to the server code.=20
>
>Hmm, why is this? Sure, there may be security problems in the server, but
>releasing it as free software will probably ensure that patches will start
>flowing in (provided the project is interesting enough to gather a
>following). I can't think of any good reasons why we must keep the server
>to ourselves -- in fact, my own ego gratification nerves are itching for
>me to release it so that I can homestead the noosphere big-time! :)

Heh.  Meant to say:

And as long as you, and/or trusted flunkies, are the only ones with access
tot he server code you are running on your site.

Don't give people open access to the code you are currently running.  Make
a copy and let them DL it for themselves, fine.  Just keep your running
copy under lock and key.  ;)

<...>
=20
>> Distrubuted does not mean "evenly distributed".  The clients can do
>> action detection, just not resolution.  As long as all clients AND the
>> server are doing detection, and the server's resolution algorithms take
>> into account the possibility of false/innacurate detections, you should
>> be safe.  You'll have to assume that any detections made by the server
>> are accurate.=20
>
>But what good is it to let clients take care of detection if the server is
>doing it as well? To me, this seems a duplication of effort. Of course,
>having the client do this might serve much like dead-reckoning in that it
>prepares the player for what is likely to happen and reduces the impact of
>latency. Nevertheless, the server must at all times be regarded as the
>authorative source of the world state.

*nod*  As I stated in another email, you could probably very easily strip
the client down to a "dumb" UI that simply passes commands on to the server
via DIS  PDUs.  Issues like this are a result of having to bend a _mostly_
(but not exclusively) PtP standard to a C/S implementation.  C/S can most
certainly be done, but you have to rethink a few things and then make them
fit back into the standard.

>> You can probably allow the client to handle modeling of motion for the
>> entity(s) it controls.  Just put some simple reality checks on the server
>> side, so that it doesn't trust the results completely. =20
>
>Right. It would be a pain to have the server compute every little bump in
>the road or swell in the surf for simulation purposes. This is better left
>to the clients. One thing to avoid like the plague is the extreme
>implementations of client-server where the client is just a stupid
>interface which echoes every little player input to the server and is fed
>the exact output with no or little computation on its part. In XPilots,
>for example, I think the server sends the entire frame to the clients --
>increasing the window size will lower your frame rate due to the increased
>network traffic, for one thing. This is clearly a Bad Thing<tm>. Of
>course, sending anything else than a pre-rendered image to the client
>paves the way considerably for client-side bots, but this is a price I am
>willing to pay. Done right, this system would be independant of the
>graphical capabilities of the client -- the client would render it as far
>as it is capable, be it (heaven forbid!) text-only or full-blown 3D
>acceleration on state-of-the-art hardware.

*nod*  DIS is pretty similar to one of the major premises behind my current
MUD project (see my section in the monthly FAQ posting, or wherever JC and
Ling have put it on the web page, if they've moved it out of the FAW=
 already).
Allowing the server to interact with lots of clients with differing
interfaces has been a goal of mine for quite a while.

>> The down side to this is that it requires that both client and server
>> have the same database to describe the environment.  You could
>> accomplish this through use of the Info PDUs, to send info about the DB
>> to the clients, if it is too large for the client to have a complete
>> copy of, or there are parts of it that you don't want the client to know
>> everything about (secret doors, etc).=20
>
>The database is a big problem. Players on unreliable and slow connections
>are generally unwilling to sit around waiting while the game initializes
>by downloading a few megs of map information, and keeping the map locally
>on the client-side is not very appealing (still, I believe it is far
>better than the first alternative). What are some other alternatives?=20
>Downloading a few chunks of the world as you go along? This is probably
>the most viable one, but might be tricky to implement and rather
>latency-prone -- especially if we are using the same database for all
>simulation type (and we are); that is, a jet fighter will cover *much*
>more ground in a second than will a lowly infantry private. That said,
>maybe there should be some notion of "database granularity" to allow the
>soldier to hide behind that little hillock while still not bogging down
>the hotshot pilot whizzing by at ten thousand feet.=20

*nod*  you may want to look at dynamic polygon reduction programs.
Assuming your DB is a large mesh of polygons, depicting x/y/z coords, and
some doodads draw on top (like trees, etc), a dynamic polygon reduction
approach might do the trick nicely.  A fighter could use a very large
granularity, and ignore all but the most prominant terrain features (look
out for that spire!), while a foot soldier would have to worry about every
bush and rock.  Of course, you could pre-calculate the DB at several
granularities, and just use the one appropriate to the scale needed (based
on player velocity, probably).

DB paging is probably also a good idea.  You could set it up so that each
page always has the same number of verticies, regardles of the granularity,
so DB updates to clients would only be affected by latency, not the
player's velocity  (imagine a Jet screaming across 100x100 meter pages, at
500 NM/hr!).

>
>If worst comes to worst, I have no problem with keeping the database on
>the client side. Some things (such as artwork, models, interfaces and
>such) will have to be kept on the client-side anyway, either by
>downloading it in-game (and preferably in the background) when moving into
>a specific theatre of war (such as WWII, Wheel of Time or Star Wars), or
>by distributing it on the CD or on a website.=20

Keeping a client side DB works fine, as long as the server is intollerant
of mistakes (IE: the client DB is hacked to remove a mountain, but the
server will not allow the hacked player to travel through the mountain.)

<...>

>I found a page containing links to the Quake network protocol at=20
><URL: http://www.activesw.com/~steve/qprotocol.html>. One of the links on
>this site (<URL: http://www.opt-sci.arizona.edu/Pandora/q2/default.asp>)
>has a good overview of the packets exchanged between the clients and the
>server. Seems that the number of different packet types exchanged between
>clients and servers is pretty low (how many PDU types are there?).=20

IEEE 1278-1 defines 27 PDU types, IEEE 1278-1a adds several more...
probably a total of 50 PDU types are defined.  A lot of the ones in 1a are
redefinitions of PDUs in 1, but with a higher level of reliability, or cut
down in size to reduce network load (EG. they add an "Entity State Update"
PDU, which contains only location, velocity, accel and heading... only
dynamic info, not mostly static info).

>Also,
>the status packet includes movement data from the three last updates
>(including the current one). Status packets are sent out once for EVER
>FRAME. This seems a whole lot, to me, considering that the framerate must
>be around 50-70 fps on high-end systems.

Ugh.  That seems a bit harsh... You'd think they'd cap it somewhere...
Though I bet they weren't counting on what some of the hardware
accelleration can do these days.=20

>Does anyone know how much I/O you can get away with on standard PC
>(client) hardware with a standard internet connection as well as on a
>dedicated server with lots of hardware and a T1 or something? I bet there
>are several ways to measure this (packets per second, throughput in bytes,
>etc), but any rate would do.

nope... all I can say is test it yourself.  Deciding what a "standard PC"
and a "standard internet connection" are is not a task I envy.  :)

-Greg