[DGD] Rewriting the main loop

[Raymond Jennings]

On Sat, Dec 1, 2018 at 6:42 AM Dread Quixadhal <quixadhal at gmail.com> wrote:

> You are correct that I don’t know how DGD internals work.  I assume that
> the main loop of the driver goes about doing each of the tasks you’ve
> listed and that each one has some limit on resource use.  For example, when
> a packet arrives from the OS, something in DGD will do a read() on the
> socket and then push the newly acquired data into a place where LPC can
> access it, likely by an apply on the LPC object associated with that socket.
>

Yes, and this apply starts as a task, formerly known as an execution
round.  Whenever an interrupt signal is processed, a callout goes off, or
new network I/O happens, it is processed as an apply to an LPC object.


> So, in other interpreters, when a task performs any potentially blocking
> operation, the driver performs the actual I/O operation and the task waits
> for it to complete.  If the driver doesn’t want to hang, it usually
> performs this operation with a timeout OR uses an asynchronous version of
> the read/write op and moves on.
>
> Part of this moving on is usually moving that task out of the run queue
> and into the wait queue, essentially freezing that task until the operation
> ultimately finishes, or is considered a failure state.  That allows the
> driver itself to continue doing other things, including other interpreted
> tasks, while the I/O is still pending.
>

Bear in mind that "task" in the context of DGD doesn't mean the same thing
as it does in OS design.


> My understanding of LPC drivers, in general, was that each object is
> basically a task, and while the driver only allows a single task to be
> loaded and running at a time (aside from Hydra), they still maintain
> state.  Most drivers make the policy decision to abort any individual
> function call that doesn’t complete in the current execution, but does that
> mean it’s utterly impossible to read data from a slow media?
>
> If anyone has access to a hardware tape drive, this could be a fun
> experiment.  Use a loop device to mount a filesystem on a tape drive (with
> a tape mounted, of course).  Now, try to start DGD and have it do basic
> file I/O.  Because a tape isn’t random access, this will take a very long
> time as it spins the tape back and forth, but it should illustrate the
> idea.  A task which only needs network I/O (such as player output) should
> run along at a normal clip, while a task needing disk I/O will crawl as the
> tape has to seek back and forth.
>

I digress.
>
> In the case of SIGTERM, you’re intending to shutdown the driver.  In that
> case, it doesn’t matter if it takes priority over other tasks, because you
> can’t know exactly what’s happening at the instant the signal is sent.
> Furthermore, you probably want the driver to shut down gracefully,
> otherwise you would have used SIGKILL.  If you prioritize this over file
> I/O, for example, you might corrupt a file being written to.
>

The only thing being prioritized is that DGD is injecting the interrupt
event before it considers injecting events for network I/O or callouts.
How LPC responds depends on how the driver object is coded.


> Also, what happens if you have an LPC hook on this signal?  I don’t know
> if you can do this in DGD, but in other drivers, you can assign LPC objects
> to catch signals and dispatch a function to deal with it.  This might
> include sending a “Oops, going down!” message to all the users before
> disconnecting them.
>

Interestingly enough an LPC hook is mandatory in this case.  If an attempt
is made to call a driver hook that doesn't exist, it will cause DGD to
crash with a fatal error.  In this case it's considered a fundamental API
violation.

What I’m trying to get at here is, that I know WHAT you’re trying to do.
> But I don’t know WHY.  You’ve said that you consider these priorities
> important, but I fail to see WHY you consider them important, and what the
> ultimate goal is for such a drastic change.  If it’s just to keep the
> driver from feeling sluggish while it’s doing swapping operations, can’t
> you just limit the number of objects swapped in or out on any given pass?
> ____________________________________________
> https://mail.dworkin.nl/mailman/listinfo/dgd