[DGD] Changing connect() (network package)

[Shentino]

> This works in DGD/MP (which knows about threads), but not in DGD
> (which doesn't).  It's not something you can solve with an #ifdef.

Yes, I AM talking about MP injection of ER's.

DGD is irrelevant.  SP DGD is so friggin simple on account of single
threadedness it would be trivial for someone to add whatever they feel
like.

I'm only talking about MP.  I don't care about plain DGD anymore, I
gave up THAT battle a long time ago ;).

And from what I gather, injecting my own ER's is about on the same
level as what MP is already doing for itself.

Not to mention that if I wanted to interoperate with OTHER extensions
that also have the same needs...if I have to patch MPo inject ER's,
and they do as well, I'm almost certainly going to make a hash of MP's
source code as my extension and theirs fight over making almost, but
not quite, the same modifications to MP to inject arbitrary ER's.

> A more accurate analogy would be, "this car does not have a builtin
> phone, because using a phone in a car increases the probability of
> an accident."  If you want to use your own phone in the car I provide,
> that's up to you.  If you make a hole in the dashboard to insert your
> phone, the warranty is void and you'll have to do your own post-mortem
> analysis.

Ah, but in this case, the car is a tightly designed model that is
inherently hostile to the addition of accessories, handheld OR
dashboard drilled.

Suppose that the car has some sort of internal interface, such as GPS
or internet with the city street systems, that my own phone would
interfere with?  In that case, using my own phone would likely be a
"warranty voiding" action as well, even if I don't futz with the car's
parts.

If the manufacturer were cooperative, there could be a USB port on the
dashboard, whereby car-compatible accessories could plug in, and the
car's own electronics could, pardon the pun, play traffic cop as
needed to give the accessory the services it requires WITHOUT
compromising the integrity of the car's operation.  The accessory
would be the only one worrying about what to send where, what number
to dial, or what street light to turn on or off.  All the car would be
doing is routing the signals to and from the internet/cellphone
waves/streets and making sure that things stayes safe.  In effect, the
car would serve as a router.

Back to MP.

Extensions handling external events, ASSUMING they can do so without
interfering with MP's operation, would require a means of interacting
with the LPC side if they were to be anything other than silent doers
of whatever LPC tells them to do.  Feedback would require some sort of
LPC notification if it's asynchronous.  Unless LPC is to be expected
to constantly poll, which is inefficient.

In the case of networking, OR sql, OR distributed computing, OR
anything else that would both be useful AND not step on DGD's toes,
for example.

I concede that itimers may wreak havoc...unless of course I install my
own signal handler, which would simply alert MP that my extension
needs to inject an ER.  As long as I don't usurp any signals that MP
is already using (i.e. SIGPIPE, SIGTERM), my extension should be able
to avoid getting MP's own code caught in a mess.  Signal handlers are
shared, and my own code would just set a flag or call a function or
otherwise yell out "Yo MP, I need to start an exec round", which
request would be returned as soon as MP's thread was ready to deal
with it.  Sorta like a deferred IRQ handler.  My extension would not
get to inject an ER until MP was ready to accept one.

Or perhaps, my code would just take note of it, and respond to the
finish() callback whenever MP was done processing an ER.  All my
extension would need is first a way to inject an ER, and second to get
notified whenever MP could do so.  If the ER would simply be queued
and dealt with at a time of MP's own choosing I'd be perfectly happy
with that.

> Itimer signals interrupt system calls and wreak havoc with DGD's internal
> design.

As far as system call interruption, I have two answers for that:

1) As far as I know, you can tell the OS that you want to
automatically retry certain operations on a signal.
2) I'd actually be surprised if properly written code would break from
getting an EINTR signal.  POSIX specs state that ANY system call can
return that, and official recommendations state that the operation
should be retried if it is still relevant.

I don't mean to be offensive, but code breaking on EINTR (if that's
what you're talking about) sounds like non-compliance with POSIX specs
anyway.  If that's NOT what you are talking about, then there's almost
certainly something about MP that I don't know about.

I think there's even a RETRYTEMP macro or something that automatically
retries a system call whenever it returns a temporary failure.

> This step has a solution, but of
> course I already know how it's going to end :-)

If your talking specifically about the networking package, I already
conceded on that point :)

> I veto itimers.

Ah, true, but my extension would do the itimer thing, not DGD.

Even if itimers couldn't be done through a pristine, virgin MP source,
having ER injection as an extension feature, would make it easier to
avoid having to patch the crap out of the source.  ER injection is the
crux of the problem for both itimers and networking.  Unlike other
bells and whistles, it is very difficult for external code to do this
without making very intrusive patches into MP's logic.  Hence, the
need for this as an expansion of the extension interface.

The model I have in mind is this:

At any time, an extension can tell MP's core "yo, I need to do
something".  This might be doable through a flag, a function, or maybe
even automatically through "finish" (I forget the name) whenever MP
finishes up with a thread.

Once my extension gets serviced, it injects an ER, in much the same
manner as MP already does whenever a new connection arrives, new input
arrives on an existing connection, or a callout goes off.  If MP can
inject an ER, then an extension should be granted the same privileges.

Now, once the ER finishes, a few things could happen:

If it was started by MP's own internal mechanisms (new connection, new
input on native connection, callout), then as you mentioned, hard
coded switch statements would take care of rescheduling or
cancellation.

If, however, the ER was injected by an extension, then, commit or
fail, MP would notice it didn't start the ER, and would just make use
of a callback supplied by the extension that injected the ER in the
first place.  The extension would decide how to handle this, as
opposed to an inline switch statement that MP would be using for its
own ER's.  Perhaps the value returned by the callback could be used.

At MP startup, the extension_init could take care of telling MP what
sorts of events the extension is planning to deal with, and MP would
simply stash that information for later for when an extension wishes
to inject such an event.

The extension would be the only one worrying about what ER to inject
when, as well as how to handle a reschedule, a cancellation, or a
commitment.

MP would provide the mechanism, but the extension would provide the policy.