[DGD] Object upgrading

[Raymond Jennings]

On Mon, Jul 2, 2018 at 2:31 AM Felix A. Croes <felix at dworkin.nl> wrote:
>
> Raymond Jennings <shentino at gmail.com> wrote:
>
> > What's the best way to make sure all objects get upgraded?
> >
> > Right now, I have a tight loop in ObjectD that atomically calls
> > call_touch on all clones of a patchable object, and then leaves a
> > background callout in PatchD to eventually nudge them all.
> >
> > Is this the best way to do it?
>
> The problem to solve with object upgrading is that an unbounded number
> of objects has to be changed, and changing them all in one task would,
> at best, be extremely inefficient, and at worst would crash the server
> by running out of memory.

Oh believe me I already looked to call_touch as a solution to this.

> DGD's unique solution is the ability to set a trigger for an object that
> will be called before the object can be used in any manner, which will
> not load that object in memory.  This will guarantee that each affected
> object can be JIT upgraded before it is used.

Obvious, and already done.

> Of course, this will only work as long as discovering the set of affected
> objects does not also load them in memory.  Furthermore, you have to be
> careful that you don't touch any of these objects from the same task
> that performs the recompilation, or the upgrade will be triggered before
> it can properly take place.

Already took care of this gotcha.

> Once the recompilation and trigger-setting has taken place, you can choose
> to either perform the upgrade at leasure in a way that doesn't load too
> many objects in memory at once, or you can leave the object as-is and
> perform the upgrade whenever the object is used again.  In the latter case,
> when an object has been upgraded more than once, it must be able to perform
> a mult-stage upgrade in a single call.

Already doing this.

Presently, PatchD maintains a database of objects that need to be
patched, and the patching process takes each object on this list, and
applies the patches while removing the object from the database after
it's patched.  I also have TouchD maintaining a list of objects that
need to be "touched", usually objects that are flagged with
call_touch, on which I call _F_dummy.

My apologies for not properly emphasizing what steps I've already taken.

So, just to be clear, this is what I do

* When an object is compiled, I ask the initd responsible for it for
its patcher function and make note of it
* When a non inheritable object is compiled, I also check its
inheritance tree for patcher functions declared by its ancestors
(information collected in the previous step)

After doing the paperwork for patchers, when a non inheritable object
is compiled, the next thing I do is atomically enter a loop to
call_touch the object and all of its clones so that the next attempt
to access them will get it pulled over for a JIT patch.

Once the clones are marked, the last thing I do before I end the task
is suspend the system and arrange to have the object's upgrade
function called, after which the system releases itself.

Meanwhile, PatchD is keeping a database of all objects with
outstanding patches, which is checked when an object marked by
call_touch is accessed, and if a touched object has pending patches,
the patchers are called and the object is removed from the database.

Caveats:

* Object being patched twice?  Declared by fiat via API that patcher
functions are responsible for checking things.
* Object being compiled before patches are completed?  Declared by
fiat via API that interrupting a patch will cause it to be aborted,
and it's the developer's responsibility to check for this and if
necessary prevent it.

In general, I'm declaring the object's programmer to be responsible
for handling the upgrade process.  It helps if the patcher function is
idempotent, but the core is that I'm asking the initd of the inherited
object for the patcher function to call when its inheriter is
recompiled.

Given the above, can you (or anyone else, really, which is why I
posted this to the list) spot any issues (no pun intended)?

My own concern is that the most obstructive part of the process is the
loop that marks all the clones.  Actually finding all the clones is a
bit of a chore.  For the time being I'm using a loop that does a
find_object sweep through the object table.



> Regards,
> Felix Croes
> ____________________________________________
> https://mail.dworkin.nl/mailman/listinfo/dgd