effekt
effekt copied to clipboard
Adding a separate lexer
This PR aims to speed up the parsing phase of the compiler, as discussed in #412. Since we would want to have a separate lexer regardless of the possibility of switching to a different parsing library or tuning the current implementation for speed, this should be useful.
The lexer is handwritten, and I took care to make it reasonably fast (of course, I am happy to hear your feedback). The reason for a handwritten lexer are of course speed, however, at a loss of ease of making changes. Though, since I would not expect the keywords, syntax of string and number literals and those for comments to change anytime soon, this should be negligible. Also, we can offer better error messages for invalid syntax than with current implementation.
unterminated string example
def main() = {
val x = "hello
()
}
>>> Expected a statement
TODO
- [X] Have a working lexer for Effekt's syntax
- [X] Cover basics functionality with tests
- [X] Add support for quoted strings / string interpolation:
"result: ${ xs.map { x => x + 1 } }"
- [ ] Integrate lexer into current parser by making the current parser implementation input agnostic -- first merge changes into kiama?