[DGD]Loading a file into an array

[abigail at delanet.com]

On Mon, Jun 19, 2000 at 12:37:01PM -0700, Par Winzell wrote:
> 
> DGD has a number of fixed and floating limitations. The 64K length limit
> on strings is fixed in the source. The maximum array size is settable in
> the configuration file. While occasionally irritating, these limits are
> really good ideas.
> 
> One reason is that there are plenty of O(N) array/string functionality
> tied to simple operators, and it would be tacky of the driver to freeze
> for 3 seconds simply because you're attempting to modify the 19th byte
> of a 7-megabyte string (which would prompt a copy), or add "foo" to the
> beginning of it.

I'm not so sure whether that's a reasonable reasoning. If you can't
have 7 Mb strings, and make a different datastructure to represent it,
you'd still need to at least 7 Mb if you need a copy. A user defined
data structure is probably going to slow down the driver for more than
3 seconds that way.

> Worse, code of the type,
> 
>   str = "-";
>   for (i = 0; i < N; i ++) {
>     str += "courtney-";
>   }
> 
> which looks very innocent is actually O(N^2) and thus deserves an upper
> limit.

Or a different implementation of strings. ;-)

> I've found in the past that the single most important trick to scaling
> up to thousands of users is to eliminate every single O(N^2) bottleneck
> which means using either mappings instead of arrays, or writing a fast
> array (or string) addition implementation in LPC which keeps a "current
> write index" variable to remember where it's writing, and only extends
> the array/string in large chunks. For example,
> 
> 
> 
> This is a very simple example, and in fact I have not compiled it, so
> treat it as basic idea only. My hope is that this kind of 'tail buffer'
> will make its way into the driver one day, rendering O(N^2) string and
> array operations O(N) at some cost in space.


That's the approach Perl takes for strings, arrays and hashes.



Abigail

List config page:  http://list.imaginary.com/mailman/listinfo/dgd