What is tinylex? A tiny lexical analyser generator

How to implement? I refer to Allen Holub's book 'Compiler design in C' and write it by myself.

Format of the input file (see test/colour.l for example, which is a C syntax higlighter):

%{ < code block for definition or macro > %} < Macro Definition: macroname macrotext > < e.g: digit [0-9] > < recursive macro definition is not allowed.> %% < Rules: regularexp action string > < e.g: {digit}+ printf("%d", yytext); > < only one-line action string is allowed. > %% < code block for main function >

Usage: compile:

make

[generate tinylex and egrep_like ]

./tinylex -p lex.yy.part.c {source script}.l -o {generated ouput}.c

test:

make test

[see the output information]

debug:

make debug

[generate many debug programes, such as nfa]

./nfa test/common_test.l

[output NFAS information according to
 common_test.l script (regular expresion for testing) ]

TODO:

1. bugfix: using grep -C 2 -n FIXME *.c to get bug information
2. make tinylex compatible with flex/lex. (Tinylex is only subset of flex/lex now. And many auxiliary functions or flags aren't supported, such as yywrap/yymore/yyrestart)
3. support multi-line accept action string (Maybe I need a internal C parser.)
4. support recursive macro definition

Author: Xiaochen Wang [email protected]