[DGD] DGD and *dbm

[Shentino]

On Tue, Jul 29, 2008 at 5:09 AM,  <bart at wotf.org> wrote:
> On Mon, 28 Jul 2008 10:19:07 -0700, Shentino wrote
>
>> Before I continue my ranting about wanting to save Dworkin work, I
>> should reveal two facts I discovered during experimentation, on a
>> crazy hunch.
>
> I think the argument has been made.
>
>>
>> 1.  Kfuns are a scarce resource
>>
>> The opcode that triggers a kfun call has only 8 bits of operand,
>> which means, that there can only be at most 256 kfuns in the driver.
>>  Kfuns are specified by 1-byte numbers, not names.
>>
>> Even the +, -, <<, etc are kfuns.
>
> Enabling the dump_function/object efuns told me this as well. Some errors give
> a hint at this also (invalid argument x to kfun + for example)
>
> ...
>
>>
>> 2.  Kfuns are a non-renewable resource
>>
>> Once added, either through the extension mechanism or through
>> Dworkin's own programming, a kfun can never be removed.  The list of
>> kfuns is stored in the statedump, and if you try to restore a
>> statedump with a driver that doesn't have all the kfuns, the driver
>> will go "wtf is this kfun?  I don't have a friggin clue what that one
>> does, so I'm not going to load this alien dump".  Even recompiling
>> initd to remove all trace of usage of "void kfun foobar()" didn't
>> help.
>
> Heh, yes, I've seen that one, and it makes perfect sense to work as it does.
> This is one thing that makes it very difficult for me to test things against
> vanilla DGD since my statedumps all have kfuns like connect() etc :)
>
>>
>> Basically, this means that every kfun added that isn't used by a
>> substantial portion of the user base is a permanent and irrevocable
>> waste.
>>
>
> Which means to me that a subsystem such as the dbm one I'm looking at, should
> implement at most a few kfuns, and those should be reusable. Preferably, one
> kfun that based on its arguments can figure out what to do.
>
> For the *dbm idea, I have been trying to see if this can be implemented as a
> variable type instead (mapping semantics, different underlying storage), but
> this doesn't seem to remove the need for at least one kfun (to 'tie' a
> database to such a variable, and possibly, to break that tie)
>
>> Already, we have 3 such kfuns.  hash_md5, hash_sha1, and hash_passwd.
>>
>> These two are probably why encrypt and decrypt are two kfuns that are
>> in a position to handle ANY encryption algorithm for LPC.
>
> That is a better solution, yes.
>
>>
>> Now, onto beefing up the extension.
>>
>> My position is not to save myself work at the expense of felix's.
>> Rather, I see an extension interface beefup as potentially powerful
>> leverage, whereby some work by felix would save MANY people lots of
>> effort.  In fact, if the extension interface is good enough, we may
>> even be able to have multiple extensions SIMULTANEOUSLY.
>
> In theory yes, and this is where I took issue also earlier on. That said,
> creating such an interface is far from simple, if at all possible. As Noah
> pointed out, this isn't a small thing to ask at all, rather, it is a very
> complex and intensive thing.
>
> Besides the 'issues' for a single extension (how to provide all the hooks and
> interfaces it needs), you also may have to be able to share a 'generic' kfun
> between extensions (this looks somewhat similar to verb handling in a mud..)
>
>>
>> Patching the driver to add networking or database is a pain by
>> itself.
>
> Its not that bad in itself once you are familiar with some aspects of DGD's
> internals. Its also highly educative.
>
>> Can you imagine how painful it would be to support BOTH at
>> the same time?
>
> I don't have to imagine, I already know :P
>
>>
>> I foresee that two patches, each of which directly modify the SAME
>> SOURCE CODE, are going to get into a big fight if they are applied
>> simultaneously.
>
> Which is why I wrote earlier that I'll very likely end up creating a common
> interface for places where they touch the same code. That should at least make
> patching relatively simple.
>

I would suggest that when MP comes around, you provide a way for ER
(execution round)'s to be injected with pretty much the same latitude
that MP's own native code would enjoy.  Having an extension define a
new ER "class", and then make the extension provide callbacks for MP
to call in case something happens, such as "The ER has
committed/rolled back/been cancelled".

You would probably need to see the actual code to know more, since I
can only speak based on theory, supported by what I've already
observed of SP's behavior and combining that with the knowledge that
MP will be almost completely backward compatible with SP, at least on
the LPC visible side.

Naturally, I would rather Dworkin do this on account of him being in
the unique position of the one with the most knowledge of how MP is
supposed to work, and therefore, the one who is least likely to err.
However, and especially on account of his most recent message to this
ML, I wouldn't be the least bit unsympathetic if he has more important
things to worry about, especially considering that there already is a
clean alternative.  But (see stuff below about irreversiblity)

I probably owe him an apology, in fact.

> ...
>
>>
>> The aforementioned "irreversibility" of expending 1 of a limited of
>> 256 slots makes me hesitant to ask for a kfun however.  The LPCSSH
>> stuff, while overhead prone, would probably stress forward
>> compatibility the least.
>
> If it can reasonably be solved in LPC, that is the prefered solution anyway.
>

The encrypt and decrypt and asn stuff being implemented as kfuns (in
all likelihood) specifically to support LPCSSH speaks volumes about
both Dworkin's preferences as well as its viability.  The technical
arguments are alone enough.

Seriously though, the complete and utter irreversibility of adding a
new kfun to the driver makes me cringe away from the extension
interface very strongly.  When the "crazy hunch" dawned into reality,
it was a big fat wet towel hitting my face with the sound of a "DUH!".

I would guess that Dworkin's support for encryption and ASN (which
seem to be specifically for LPCSSH) is already plenty of work. ;)

> Bart.
> ---
> Created with Open WebMail at http://www.bartsplace.net/
> Read my weblog at http://soapbox.bartsplace.net/
>
> ___________________________________________
> https://mail.dworkin.nl/mailman/listinfo/dgd
>