[DGD] Changing connect() (network package)

[bart at wotf.org]

On Mon, 31 Dec 2007 10:29:44 -0800, Shentino wrote
> Wow...what a feud :)

Haha, welcome to discussing networking in DGD :)
> 
> As far as "boo on net package", all of Dworkin's arguments are true,
> however, an external daemon may cause the same problems.  And here,
>  we have some arbitrary "3rd party code" that isn't even benefiting from
> the same coding expertise that went into DGD & DGD/MP in the first
> place.

Yes, and on top of that, if trusting people to write proper and secure LPC
code for networking is an issue, then I'd argue that doing so in a less safe
language would just be a bigger issue.

Its one reason why I personally think an integrated package is the better
solution, abeit with the issue of using undocumented interfaces etc, it would
be a lot better if there is a clean way to integrate it :)

That said, the networking code in current versions of DGD isn't so amazingly
complex that the unofficial interfaces are a real problem right now, and while
Dworkin may be unwilling to actually add or support a networking package, he
has shown to be quite willing in helping people like you and me explore the
internals of the driver way beyond the official interfaces, which makes it
quite feasable to maintain such a package still.

> 
> Also, handling ANYTHING outside of DGD introduces overhead in the 
> form of maintenance, signalling, context switching, and you have yet 
> another process competing with DGD for the attention of the OS.  You 
> now have TWO queues that can backlog, instead of just one, for every 
> connection.  With an external "network daemon" it has to deal with 
> I/O with the external world.  THEN it must also relay to and fro 
> with DGD itself, which itself introduces networking overhead, even 
> if it's just from localhost to localhost.

this is true, but I think you should give Par's idea some merrit.

A daemon could take away a lot of the nasty details of networking from DGD.

For example, there are many network protocols that produce a bytestream in
which the beginning of a network packet and the beginning of a bit of data on
the protocol level don't have to match, and often won't. Because of this, you
can't just assume to find protocol specific information only at fixed
locations in a packet, they can be anywhere in the bytestream.

A nice example of such a protocol is telnet. Yes, you do often get away by
assuming that telnet commands start on packet boundaries, but the RFC very
explicitly says to not count on it, and indeed, when you try to handle
something slightly more complex (remote editing for example) the assumption
fails at least at times.

The only way to deal properly with such protocols is scanning them byte for
byte. While DGD can quite do this, LPC is very likely not the most efficient
way to do this, and when you are dealing with any entity > MAX_STRLEN, you
need to put it a fair bit of extra work to make it work at all.

When doing the same thing in say JAVA or C++ or such, that would be far less
of an issue, and it could then hand the data to DGD in easy to 'digest' amounts.

I could imagine you running a cluster of game servers to support a really
large gameworld. There are a few ways to do this, which are interesting to
debate, but not so important for this discussion. What is important is what
you point at, you want those servers to talk to eachother so that they can act
as parts of a larger world.

You could have all those servers talk to eachother in a mesh like setup, you
could also opt for some more centralized 'coordination' nodes, and other nodes
that deal with player sessions. In such a setup, DGD would always be talking
to a coordinating node, which would broadcast/multicast relevant information
to other nodes. Instead of each instance of DGD having to keep track of what
it delivered where, thats offloaded to the coordination daemon.

I see some advantages to both approaches, and don't think you can easily claim
one to be better then the other in all cases.

> 
> As far as the "solution" being "outlaw outbound connections", it's
> pretty much the same thing as saying "nobody should drive a car
> because you could run someone over".  It makes sense in the case of a
> game that is running on a single server, but what about some sort of
> P2P deal?

As I pointed out, even there DGD does initiate outbound traffic, abeit in a
very limited way and out of reach for your LPC code.

Obviously, I completely agree with your argument :)

> 
> At any rate, DGD lends itself quite easily to purposes OTHER than
> games.  Heck, didn't iChat use it to capture like 80 percent of the
> market in only 6 months or so?  If I were of a mind to, DGD with
> networking capabilities could make a very kick-arse IRCd.

Definitely, and when looking back at that coordination node idea I was just
talking about, I'd again use DGD to implement that, despite some of the issues
(large packets etc) since it is a very robust environment that with a proper
kernel enforces coding and security standards much better then say a Unix
environment, its extremely lightweight (that would be my main objection
against using JAVA for this), and you can isolate those who have to deal with
it inside the LPC environment.

Not having to give people shell access is a very big one for me in such
scenarios (and is part of why I think the "if you are interested in outgoing
connections in DGD you can't be interested in security" isn't always a very
good one, I rather trust people with outbound connectivity before trusting
them with full shell access.

<snip>

> Using an external daemon for this could at best introduce massive
> overhead in the form of doing nothing but being a redirector.
> Networking code that resides in the DGD (or DGD/MP) process can do a
> MUCH better job.

It could, or the daemon could, that depends on the kind of network you
implement between your nodes, and on the protocol you use for it.

> 
> But in the general case, a network interface for DGD would be quite useful.
> 
> And if that's not enough, there are plenty of other things besides
> networking that could benefit from arbitrary injection of ER's.  Such
> as an asynchronous "reset" interface, or perhaps a "cleanup" 
> mechanism that goes around firing off garbage collection.

I agree, and while I have no illusions whatsoever about Dworkin ever adding
outgoing network connections to DGD, I do somewhat hope he can be convinced it
is a good idea to facilitate such extensions :)

--
Created with Open WebMail at http://www.bartsplace.net/
Read my weblog at http://soapbox.bartsplace.net/