[DGD] Pathological memory allocation?

[Raymond Jennings]

Yeah but I checked my code (and double checked it after the replies
here).  The data structures in question are as follows:

An array of around 80 elements
    Each of which has strings that are routinely no longer than 300
characters each

A linked list
    ({ first, last })
    Each node is: ({ prev, string, next })

The length of this list routinely reaches into several thousand but
each individual string is about 4K in size.  I have a background
0-callout that is steadily popping the back of the list.

Given that the sizes are capped at a few k I'm still perplexed that
slack space is getting up into the multiple gigabytes.
On Tue, Oct 23, 2018 at 9:52 PM Felix A. Croes <felix at dworkin.nl> wrote:
>
> Dread Quixadhal <quixadhal at gmail.com> wrote:
>
> > I assume that when you do x = x + ({ 1 }), the engine will look for a memory block that's the size of x + 1 elements, and if it doesn't find one, it will allocate a new one and mark the old one as available.
> >
> > So, if you do a loop where you keep adding elements one-at-a-time, you'll end up using a very large amount of system memory, as there will be lots of N-1 sized chunks sitting around available, but unused.
>
> N-1 sized, and N-2 sized, and...  memory wasted is quadratic.
>
> This is a general problem, it also exists in Java, for example.  As in
> Java, in the cloud library I added a StringBuffer class, where small
> pieces can repeatedly be added without wasting a lot of memory for
> intermediates.
>
> Regards,
> Felix Croes
> ____________________________________________
> https://mail.dworkin.nl/mailman/listinfo/dgd