[DGD] parse_string

[pete at ana.sk]

On 25 Jul 2001, at 21:26, Felix A. Croes wrote:

> pete at ana.sk wrote:
> 
> >[...]
> > First, it looks like there are some problems with parse_string when 
> > same grammar is used many times, and grammar tokens use '\0' 
> > character in them. This is not so important because i dont think it 
> > can match '\0', but i accidentaly used it in one grammar, and after 
> > few tryes it starts parsing badly. It does work ok few time though! 
> > So there may be some other problem that is just triggered by '\0'...
> 
> Could you post an example grammar where this problem occurs?  '\0'
> should definitely be handled without trouble.

Make just simple grammar like
whitespace = /[\0- ]+/
word = /[^\0- ]+/
text : word text
text :

and try parse strings like "text 1", "text 2", "text 3"... first few times 
(mabe 10 or so) it returns ({ "text", "1" }) etc. but then it starts to 
return just ({ "text" }). But you must use only this grammar, using 
any other grammar in that object resets it and it starts to work 
again.

> > But more serious thing, it would be good to have some predefined 
> > special token that matches only at string end. I really dont like to 
> > add '\1' or something like that to string end and use it in grammar. 
> > As far as i remeber what i have learned about grammars in school 
> > (its some years ago so i forgot most:) its normal for grammar have 
> > such token.
> 
> But where does a string end -- at spaces, punctuation, C whitespace,
> non-ASCII characters, and last but not least, how about non-ASCII
> character sets?  I didn't think that parse_string should have any
> assumptions about this built into it.

I thought string end is pretty well defined, its when there are no 
more characters in string. I mean when there are no more 
charactes in string to parse, then string-end token should match

Pete

_________________________________________________________________
List config page:  http://list.imaginary.com/mailman/listinfo/dgd