antlr4
antlr4 copied to clipboard
Support "Schrödinger's token"
The CaretToken
used in ANTLRWorks 2 and GoWorks already acts as this type of token used with ANTLR 4. A generalized mechanism for handling this could be implemented by allowing the pushMode
operation to push more than one mode in parallel, creating multiple branches of the lexer. For example, a new command could be created with the following form:
pushAnyMode(PossibleModeA, PossibleModeB)
The actual determination of the mode taken during parsing is determined by the prediction algorithm at the parser at the time the potential tokens are examined.