[DGD] To token or not to token...

Sun Apr 18 21:13:52 CEST 2010

Hello list!

I have a problem I wanted to solve with the parse_string() kfun, and
could use some input, if anyone feels they could contribute.

What I want to do is to parse a HTML template with inline LPC code.
Basically, I will use '<%' and '%>' as syntax for the opening and
closing of LPC code blocks. My problem is with regards to quoted
strings. I want to parse any code blocks within HTML strings, but I do
not want to parse any code blocks within LPC strings.

Consider the following input string as an example:

<html attribute="<%write("foo");%>">
<%
    write(xml_escape("<%foo%>"));
%>
</html>

Where I would want an output something like:

({ ({STRING, "<html attribute=\""}), ({CODE, "write(\"foo\");"}),
({STRING, "\">\n", ({CODE, "\n    send(\"<%foo%>\");\n"}), ({STRING,
"\n</html>\n"}) })

Where STRING and CODE would just be some constants used for
identification later.

My understanding is that I can't use a token pattern to match string
constants here, and then things turn tricky for me...

Should I try to parse this with just the double quote as a token, and
work my way from there with the resulting parse tree? Or should I
maybe still use a full string token, but use a second parse pass to
parse the matched strings depending on where I find them?

Input appreciated if you have any!

Thanks,

Jimorie