Shahar Soel

Custom Token Patterns - better state access

1

- [ ] Expose and allow mutating location state for custom Tokens. - [ ] Investigate performance implications of tracking newlines using custom Tokens. - avoids rescanning the text. -...

The error message could contain some lines before and after and error and also include a little marker on where the error is. See: https://prettier.io/playground/#N4Igxg9gdgLgprEAuEAzArlMMCW0AEAFnADYkQDqEATgCYAUAlPsADpT74BuAhtfgA98AXnwBWdpwgQADgGd8ASXy0ctfDhj4eAcx44ok7n3wBPEfgBs7AL4gANCFm5oc5KD7UIAdwAKfBDcUHi4INQcQACNqHjAAazgYAGUZWIMdZBhqdDhHQhgAWxIKQk04OVSwOCTAzRwuTVNkcDk3RwM5OGoYXxidAp5kVB4STscAKzkBACEY+MSkngK4ABkDOCGRsZBJgST0kjgARXQIeE3R3JBU6k7qZsieSNNyKAiZagMYCjUYQmQABwABkcHwgnQoMRkzQ+5S6XA2jmocAAjugcMjeroBhdtp0CjhMtkrnIDsdTuckMNLo4YE8frQ-sgAEy0mI4EjpADCEAKOJQUGgiJA6E6ABUnkFqZ0bDYgA

So Chevrotain users could add a single watch expression to assist in the debugging of grammar implementation issues. ```typescript export interface debugInfo { tokenIdx: number nextToken: IToken followingText: string isBacktracking:...

Custom Lexer Adapters

7

- [X] Interface. - [X] Implementation for default TokenVector based Adapter. - [ ] JS Docs. - [x] Performance check --> **Regressions**. - [ ] Examples: * Just In Time...

The ILexerConfig requires two very similar properties to handle custom line terminators https://sap.github.io/chevrotain/documentation/3_6_1/interfaces/ilexerconfig.html With a breaking change we can probably only accept a list of strings or charCodes to be...

API for Tokens Emitting in custom token patterns

3

Originated from https://github.com/SAP/chevrotain/issues/373 and https://github.com/SAP/chevrotain/issues/414

Investigate better backtracking optimizations.

1

## Background The Java-Parser is implemented in a style as closely resembling the official specifications. This is done to increase confidence in it's correctness and ease the maintainability/upgradability when new...

Create Demos/Playground

6

Examples: - https://github.com/SAP/ui5-typescript/tree/master/demos/demo-js - https://github.com/bd82/toml-tools/tree/master/test-packages/test-prettier-plugin-toml Note both these mini example packages include links to gitpod.io workspaces so no needs to clone and setup an env locally 😄

Parser Documentation

1

- [ ] Optimized backtracking - [ ] Design goals - [ ] How to debug