[DGD] DGD and *dbm

[Shentino]

Speaking of "even non-atomic threads getting rolled back under MP",
how are we going to consistently be able to access the filesystem?

I remember a "recent" change to allow DGD to interact with named
pipes, sockets, and other such things that have *read side effects*.

I'm assuming that FS access will become a critical section of some
sort, or buffered like user object I/O, but I'd like to hear it from
the horse's mouth exactly how DGD/MP will behave from the filesystem's
point of view.

Alas, filesystem writes are illegal under atomic code, but what about
non-atomic code that happens to be rescheduled?

I do have interest in finding out, since I was planning on writing an
external daemon to handle networking, and it would make heavy use of
named sockets and/or pipes to allow DGD to read/write
commands/status/data to/from said daemon.

In fact, I already wrote a shell daemon to implement a "system()" type
call, simply by dumping the command into a fifo, with the shell daemon
parked on the other side.

Nothing personal about competing with the new net package, I just plan
to cater to those who are shy about using patched drivers. ;)

But anyway, the FS side concerns me.
On Mon, Jul 21, 2008 at 5:54 AM,  <bart at wotf.org> wrote:
> On Mon, 21 Jul 2008 14:18:33 +0200, Felix A. Croes wrote
>> bart at wotf.org wrote:
>>
>> > The idea I'm playing with is to add support for one of the *dbm packages (bdb,
>> > gdbm, tdbm etc). This would serve a number of purposes, including storage of
>> > huge amounts of (potentially large) key:value pairs, and possibly, use it to
>> > share data with external applications.
>> >
>> > The questions I have:
>> > Because any such dbm might take a fair amount of time to return some results,
>> > I'm considering a callback mechanism to return results. This breaks
>> > continuity, ie, you have to request data, and then wait for the callback
>> > instead of being able to do something like var = dbm_get(key). I'm not sure if
>> > this is really needed when requesting a single key:value pair tho. Any
>> > experience with this?
>>
>> I have even more questions:
>>
>> Will C-level calls to the database be handled by DGD's interpreter,
>> or will the database run in a different process or thread?
>
> Initially, by DGD's interpreter, possibly as a seperate process to prevent it
> from blocking the interpreter while the db is busy.
>
> If this proves unworkable (I could see some reasons for this), it will be
> easier to move the entire thing into a seperate process and use a connection
> to communicate with it. That is however the thing I'd like to avoid (it is
> probably desirable for larger, more serious setups, but having it integrated
> into the driver is a lot easier for my 'hobbymud' setup. It would also be a
> lot 'cleaner' in that it wouldn't touch any DGD internals I imagine, and yes,
> that is the networking discussion all over again)
>
>>
>> Will the database play fair with atomic functions?  How will an
>> atomic rollback affect it?
>
> At least bdb and tdbm support a form of transactions, a commit could happen
> after DGD completed the atomic execution with no rollback. I haven't looked at
> this too closely yet, it is an important consideration however. As a worst
> case workaround, enforce that the dbm related kfuns can't be used from within
> atomic code (not feasable for DGD/MP I suppose)
>
>>
>> Do you intend it to work with DGD/MP in the future, where even non-
>> atomic code can be rolled back?
>
> I'd love to, but for now I can only target what I can actually work with :)
> In all seriousness, I don't expect such a thing to directly work on DGD/MP
> from all the things you have explained about it so far. Support for DGD/MP
> will be similar to the network package.. Not at all right now, but once I can
> get my hands dirty with MP, I'd like to see if I can port the code to it. I
> expect this to be a fair amount of work also.
>
>>
>> > Since there can be more key:value pairs in such a dbm then fit in an array or
>> > mapping, it won't be possible to get a list of all keys or values directly, I
>> > am considering returning a range (of max max_array elements), based on
>> > start/end arguments (ie, arr = dbm_keys(x,y); ). Any thoughts on this?
>>
>> Have you considered arrays of arrays (of arrays...) of results?
>
> Yes. I'm not sure which way would be more 'convenient' to LPC coders, its
> definitely an option, it would just take a bit longer to produce the result.
>
>>
>> What sort of information are you allowing to be stored in the database?
>> Strings only, everything but objects, or anything whatsoever?
>
> Everything but objects (Everything encoded in the format used by save_object)
>
> Bart.
> --
> Created with Open WebMail at http://www.bartsplace.net/
> Read my weblog at http://soapbox.bartsplace.net/
>
> ___________________________________________
> https://mail.dworkin.nl/mailman/listinfo/dgd
>