antlr4 icon indicating copy to clipboard operation
antlr4 copied to clipboard

Refactoring of grammar literals parsing

Open KvanTTT opened this issue 1 year ago • 5 comments

Code for parsing grammar string literals 'abc' and grammar char set [abc] have a lot of in common and I unified it. I extracted the functionality into GrammarLiteralParser that is responsible for parsing from string literals and char sets. It has the following methods:

  • parseStringFromStringLiteral
  • parseCharFromStringLiteral (returns CharParseResult)
  • parseChar (returns CharParseResult)
  • parseNextChar (returns CharParseResult)

CharParseResult can be either INVALID or CODE_POINT or PROPERTY. PROPERTY is supported only by char set.

Also, I unraveled ANTLR tool and ANTLR runtime functionality of char processing.

KvanTTT avatar Jul 09 '22 11:07 KvanTTT

@parrt it's also ready. Tool tests will be fine after #3779

KvanTTT avatar Jul 09 '22 11:07 KvanTTT

Hmm...pretty risky modifying the ATN construction...

parrt avatar Jul 09 '22 16:07 parrt

Can you help me understand the prime motivation here? There's a lot to review and a lot of changes that could affect allow users, because it touches the tool itself.

parrt avatar Jul 09 '22 16:07 parrt

Can you help me understand the prime motivation here? There's a lot to review and a lot of changes that could affect allow users, because it touches the tool itself.

I decided to rebase and suggest my old commits since they exist, remove code duplication (more removed lines than added) and fix inaccurancy in error messages (see tests). Our test coverage is good enough especially for the tool. Error messages without location look akward.

I've touched the tool several times, and it almost hasn't cause any trouble.

KvanTTT avatar Jul 09 '22 17:07 KvanTTT

I'm going to leave this active as it could be useful but I'm allocating my time to antlr-as-a-service proj at moment. thanks!

parrt avatar Jul 15 '22 17:07 parrt