jflex
jflex copied to clipboard
Support returning multiple tokens in one action
It is occasionally desirable to return multiple terminals from one action. This StackOverflow question gives an example of this and a couple options for handling this with Flex. In particular, it lists this nonblock do-loop in Fortran:
DO 20 I=1, N ! line 1
DO 20 J=1, N ! line 2
! more codes
20 CONTINUE ! line 4
In that example, the last 20 serves as the end of both the first and second DO
loops. It is then desirable to emit two ENDDO
terminals when the 20 is matched.
The Flex solution seems to be to insert some code into the yylex
method that first returns from a token queue and then tries to match tokens. When you'd like to return two tokens you just return the first one and put the second one in the queue.
Does Jflex provide a way to do this or otherwise support this use case?