[DGD] parse_string question

[Felix A. Croes]

Noah Gibbs <noah_gibbs at yahoo.com> wrote:

> Then I add a 'bad token' rule.  The rule looks like
> this:
>
> bad_token = /.*/
>
> It comes after all other token rules in the file.  So
> it shouldn't affect any existing parseable grammars,
> but it should make a bunch of inputs stop giving
> "Invalid token at offset XXX" errors and start just
> returning nil.
>
> No dice.  It turns out that adding the bad token rule
> suddenly makes simple transitive verbs ("get lamp",
> "look road") stop parsing.

The bad_token matches everything, and parse_string() gives
preference to the longest possible match.  So one token
only matches "get" and another matches all of "get lamp"
as a single token, and the latter one will be considered
to be the real match.

The ordering of tokens in the grammar only comes into play
when two tokens match the same input string.  That doesn't
happen here, "get lamp" is matched only by bad_token.

Regards,
Dworkin
_________________________________________________________________
List config page:  http://list.imaginary.com/mailman/listinfo/dgd