intellij-plugin-v4

intellij-plugin-v4 copied to clipboard

Is there a way to test a rule that needs a different lexer mode?

As far as I can tell, there is no way to change the lexer mode from the plugin.

Hi, it's not supported in the current version. I'll see if the interpreter used by the plugin supports this.

Thanks. Can you point you to the interpreter used? In Python, there is a mode() function you can use on the Lexer:

istream = InputStream(s)
lexer = OverrideLexer(istream)
lexer.mode(OverrideLexer.VALUE_MODE)

Related to this: While working with modes (and having a customized output directory), the plugin left a tokens file (OverrideLexer.tokens) next to the lexer g4 file instead of in the output directory.

The plugin is using a subclass of org.antlr.v4.tool.GrammarParserInterpreter.

The tokens file is probably a bug, the plugin is not always aware of the correct output directory (e.g. in Maven/Gradle projects). If the output directory was set manually in the settings dialog, then it's definitely a bug :)

Woops sorry we're talking about a org.antlr.v4.runtime.LexerInterpreter, which extends org.antlr.v4.runtime.Lexer so it should be possible to change the mode. I just have to figure out the best way to integrate this in the current UI.

Awesome. Once you figured out a good UI for it: I think something really nice to support is also making the mode sticky per rule. Once I change the mode for a rule, it should stay like this until I change it back.

re: output directory: yes, it was set in the plugin settings.

While working with modes (and having a customized output directory), the plugin left a tokens file (OverrideLexer.tokens) next to the lexer g4 file instead of in the output directory.

Investigating. Looks like a bug actually where are you can set the gen during the parser but it's ignored by the separate lexer. https://github.com/antlr/intellij-plugin-v4/issues/548

Are we talking about setting just the starting mode for lexing? I think it would get very confusing if there's some state indicating which mode we are lexing in.

I see this is still and open issue - What would be nice is to flag the starting mode (either from the Test Rule xxx ) by using a nest popup of the lexar modes or maybe by using an option on the rule to state the preferred starting mode for testing.