checkstyle-idea
checkstyle-idea copied to clipboard
jshiell
java.lang.ClassCastException: antlr.CommonHiddenStreamToken cannot be cast to antlr.Token
With the last updates of Idea and the plugin I get following error. If I press reload config it work for an short time but then it appears again. OS: Windows IDEA: 2019.1.3
org.infernus.idea.checkstyle.exception.CheckstyleToolException: Exception was thrown while processing Foo.java
at org.infernus.idea.checkstyle.service.CheckstyleActionsImpl.executeCommand(CheckstyleActionsImpl.java:136)
at org.infernus.idea.checkstyle.service.CheckstyleActionsImpl.scan(CheckstyleActionsImpl.java:76)
at org.infernus.idea.checkstyle.checker.CheckStyleChecker.scan(CheckStyleChecker.java:45)
at org.infernus.idea.checkstyle.checker.ScanFiles.lambda$checkFiles$6(ScanFiles.java:190)
at java.util.Optional.map(Optional.java:215)
at org.infernus.idea.checkstyle.checker.ScanFiles.checkFiles(ScanFiles.java:190)
at org.infernus.idea.checkstyle.checker.ScanFiles.processFilesForModuleInfoAndScan(ScanFiles.java:149)
at org.infernus.idea.checkstyle.checker.ScanFiles.call(ScanFiles.java:75)
at org.infernus.idea.checkstyle.checker.ScanFiles.call(ScanFiles.java:31)
at com.intellij.openapi.application.impl.ApplicationImpl$2.call(ApplicationImpl.java:339)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.puppycrawl.tools.checkstyle.api.CheckstyleException: Exception was thrown while processing Foo.java
at com.puppycrawl.tools.checkstyle.Checker.processFiles(Checker.java:307)
at com.puppycrawl.tools.checkstyle.Checker.process(Checker.java:220)
at org.infernus.idea.checkstyle.service.cmd.OpScan.processAndAudit(OpScan.java:76)
at org.infernus.idea.checkstyle.service.cmd.OpScan.execute(OpScan.java:52)
at org.infernus.idea.checkstyle.service.cmd.OpScan.execute(OpScan.java:22)
at org.infernus.idea.checkstyle.service.CheckstyleActionsImpl.executeCommand(CheckstyleActionsImpl.java:130)
... 13 more
Caused by: java.lang.ClassCastException: antlr.CommonHiddenStreamToken cannot be cast to antlr.Token
at antlr.CharScanner.makeToken(CharScanner.java:173)
at com.puppycrawl.tools.checkstyle.grammar.GeneratedJavaLexer.mBLOCK_COMMENT_CONTENT(GeneratedJavaLexer.java:1258)
at com.puppycrawl.tools.checkstyle.grammar.GeneratedJavaLexer.mBLOCK_COMMENT_BEGIN(GeneratedJavaLexer.java:1199)
at com.puppycrawl.tools.checkstyle.grammar.GeneratedJavaLexer.nextToken(GeneratedJavaLexer.java:335)
at antlr.TokenStreamHiddenTokenFilter.consume(TokenStreamHiddenTokenFilter.java:38)
at antlr.TokenStreamHiddenTokenFilter.consumeFirst(TokenStreamHiddenTokenFilter.java:42)
at antlr.TokenStreamHiddenTokenFilter.nextToken(TokenStreamHiddenTokenFilter.java:122)
at antlr.TokenBuffer.fill(TokenBuffer.java:69)
at antlr.TokenBuffer.LA(TokenBuffer.java:80)
at antlr.LLkParser.LA(LLkParser.java:52)
at com.puppycrawl.tools.checkstyle.grammar.GeneratedJavaRecognizer.compilationUnit(GeneratedJavaRecognizer.java:158)
at com.puppycrawl.tools.checkstyle.JavaParser.parse(JavaParser.java:97)
at com.puppycrawl.tools.checkstyle.TreeWalker.processFiltered(TreeWalker.java:160)
at com.puppycrawl.tools.checkstyle.api.AbstractFileSetCheck.process(AbstractFileSetCheck.java:85)
at com.puppycrawl.tools.checkstyle.Checker.processFile(Checker.java:332)
at com.puppycrawl.tools.checkstyle.Checker.processFiles(Checker.java:294)
... 18 more
Does this happen with Checkstyle on the command line as well? That exception does look like it's coming out of the parser in Checkstyle rather than from the plugin.
not tested yet. but we have a job on buildsystem to use checkstyle as commandline and there are no errors. Maybe it is some library in IDE classpath which causes the conflict
It wouldn't be the first time that's happened. I certainly can't find any relevant bug in Checkstyle's list.
Could you please attach your rules file? I'll try have a look at this ASAP, although it'll probably be this weekend at the earliest. I don't have a Windows box either, but it doesn't look like a platform issue 🤞
thanks for look.
I've take a look on the release notes of checkstyle and see following: 8.20: Remove usage of antlr specific methods from our checks. Author: rnveach #6624 8.21: API: make api not depend on antlr by splitting DetailAST between implementation and interface. Author: rnveach #3417
Maybe one of them is causing it
I choosed 8.19 within settings but error still occurs.
Stupid thought - did this happen while you were editing a file, e.g. might the file have been syntacticly incorrect at the time?
Checkstyle only commit to supporting syntactically correct files - the output for anything unparsable is unpredictable, by design (i.e. it just propagates whatever happens). As such, plugin authors need to do some extra error handling to catch these cases, and every so often a new variation pops up that breaks this. Hence I'm wondering if this might be one of those cases.
Yes most of them occurs if I edit a file. And IDea saves automatically and an check could be run in background. But then the message is wrong:
java.lang.ClassCastException: antlr.CommonHiddenStreamToken cannot be cast to antlr.Token
Haven't tried as yet, this was just from thinking through the problem on my commute.
I thought this is not the main problem. Now I get the message too if I change nothing but switching between files. Sorry for that bad news
All is not yet lost - it's definitely hitting this case, having traced it, although it may be hitting another case as well.
And it's quick to fix, so I can get this out now (5.29.1) and - assuming problems continue - can investigate further when I've more time to hand.
Thanks - no rush, mind - looks like the weekend is the earliest I'll get time to have another go 😄
I've updated the plugin but a new error occurs. But this is not happen so many times like before
2019-07-11 17:31:38,534 [27599292] WARN - a.checkstyle.checker.ScanFiles - An error occurred while scanning a file.
org.infernus.idea.checkstyle.exception.CheckStylePluginParseException: Exception was thrown while processing Foo.java
at org.infernus.idea.checkstyle.service.ExceptionWrapper.wrap(ExceptionWrapper.java:34)
at org.infernus.idea.checkstyle.service.CheckstyleActionsImpl.executeCommand(CheckstyleActionsImpl.java:132)
at org.infernus.idea.checkstyle.service.CheckstyleActionsImpl.scan(CheckstyleActionsImpl.java:76)
at org.infernus.idea.checkstyle.checker.CheckStyleChecker.scan(CheckStyleChecker.java:45)
at org.infernus.idea.checkstyle.checker.ScanFiles.lambda$checkFiles$6(ScanFiles.java:190)
at java.util.Optional.map(Optional.java:215)
at org.infernus.idea.checkstyle.checker.ScanFiles.checkFiles(ScanFiles.java:190)
at org.infernus.idea.checkstyle.checker.ScanFiles.processFilesForModuleInfoAndScan(ScanFiles.java:149)
at org.infernus.idea.checkstyle.checker.ScanFiles.call(ScanFiles.java:75)
at org.infernus.idea.checkstyle.checker.ScanFiles.call(ScanFiles.java:31)
at com.intellij.openapi.application.impl.ApplicationImpl$2.call(ApplicationImpl.java:339)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.puppycrawl.tools.checkstyle.api.CheckstyleException: Exception was thrown while processing Foo.java
at com.puppycrawl.tools.checkstyle.Checker.processFiles(Checker.java:307)
at com.puppycrawl.tools.checkstyle.Checker.process(Checker.java:220)
at org.infernus.idea.checkstyle.service.cmd.OpScan.processAndAudit(OpScan.java:76)
at org.infernus.idea.checkstyle.service.cmd.OpScan.execute(OpScan.java:52)
at org.infernus.idea.checkstyle.service.cmd.OpScan.execute(OpScan.java:22)
at org.infernus.idea.checkstyle.service.CheckstyleActionsImpl.executeCommand(CheckstyleActionsImpl.java:130)
... 13 more
Caused by: java.lang.ClassCastException: antlr.CommonHiddenStreamToken cannot be cast to antlr.Token
at antlr.CharScanner.makeToken(CharScanner.java:173)
at com.puppycrawl.tools.checkstyle.grammar.GeneratedJavaLexer.mBLOCK_COMMENT_CONTENT(GeneratedJavaLexer.java:1258)
at com.puppycrawl.tools.checkstyle.grammar.GeneratedJavaLexer.mBLOCK_COMMENT_BEGIN(GeneratedJavaLexer.java:1199)
at com.puppycrawl.tools.checkstyle.grammar.GeneratedJavaLexer.nextToken(GeneratedJavaLexer.java:335)
at antlr.TokenStreamHiddenTokenFilter.consume(TokenStreamHiddenTokenFilter.java:38)
at antlr.TokenStreamHiddenTokenFilter.consumeFirst(TokenStreamHiddenTokenFilter.java:42)
at antlr.TokenStreamHiddenTokenFilter.nextToken(TokenStreamHiddenTokenFilter.java:122)
at antlr.TokenBuffer.fill(TokenBuffer.java:69)
at antlr.TokenBuffer.LA(TokenBuffer.java:80)
at antlr.LLkParser.LA(LLkParser.java:52)
at com.puppycrawl.tools.checkstyle.grammar.GeneratedJavaRecognizer.compilationUnit(GeneratedJavaRecognizer.java:158)
at com.puppycrawl.tools.checkstyle.JavaParser.parse(JavaParser.java:97)
at com.puppycrawl.tools.checkstyle.TreeWalker.processFiltered(TreeWalker.java:160)
at com.puppycrawl.tools.checkstyle.api.AbstractFileSetCheck.process(AbstractFileSetCheck.java:85)
at com.puppycrawl.tools.checkstyle.Checker.processFile(Checker.java:332)
at com.puppycrawl.tools.checkstyle.Checker.processFiles(Checker.java:294)
... 18 more
Where are you seeing the exception? That looks like the log file?
An exception will be logged - we've received an error from the plugin, and have no definite way of differentiating an exception caused by a programming error (e.g. the plugin or Checkstyle has broken) and one caused by unparsable input. Hence we'd log it, and it should be present in the event log and in the IDEA log files - but it shouldn't cause an uncaught exception (i.e. the flashing red icon in the bottom right of the IDE window).
The alternative would be to hide it, which would a) hide real errors and b) make it looks like the scan wasn't happening at all.
Hmm. Are you able to share a sample of code which produces this? I've managed to produce similar traces, but haven't managed the ClassCastException exactly yet.
I appreciate this may not be possible for confidentiality/commercial reasons, of course.
I can try to reproduce it with sample code but the original code (commercial) cannot be shared it occurs in many files. After your fix mostly before commit to SVN (with check with checkstyle)
That's perfect - that's what I was missing. I was presuming it was occurring with the real-time scan, as parse problems tend to be linked to code being edited. It doesn't look like we're handling this case for on-commit scans, as we expect the code to be parseable.
Which raises the question of what to do with this. Checkstyle itself is throwing the error, so we have a choice of either continuing to raise it as an error, or of suppressing it on the basis that the user knows what they're doing. What do you think?
And there's a third option, which I'm working on now - which is to clean up the error output so it's clear as to the cause, e.g.
Thanks I will try it monday on work
But to clarify: The first error " antlr.CommonHiddenStreamToken cannot be cast to antlr.Token" occurs during runtime checks if editing files or switching editors with valid files. After 5.29.1 the error mostly occurs during pre commit check but sometimes at runtime.
I use OpenJDK 11 and java11 sources. Maybe this is another hint.
Tested the new version on work. The error within IDE Event Log occurs if I change a file and the file is checked (either during runtime or by click 'Check Current Change List'.
17:28 The source file could not be parsed by Checkstyle
The scan failed due to a parse error: Exception was thrown while processing Foo.java
Root cause:
antlr.CommonHiddenStreamToken cannot be cast to antlr.Token
java.lang.ClassCastException: antlr.CommonHiddenStreamToken cannot be cast to antlr.Token
at antlr.CharScanner.makeToken(CharScanner.java:173)
at com.puppycrawl.tools.checkstyle.grammar.GeneratedJavaLexer.mBLOCK_COMMENT_CONTENT(GeneratedJavaLexer.java:1258)
at com.puppycrawl.tools.checkstyle.grammar.GeneratedJavaLexer.mBLOCK_COMMENT_BEGIN(GeneratedJavaLexer.java:1199)
at com.puppycrawl.tools.checkstyle.grammar.GeneratedJavaLexer.nextToken(GeneratedJavaLexer.java:335)
at antlr.TokenStreamHiddenTokenFilter.consume(TokenStreamHiddenTokenFilter.java:38)
The Checkstyle Window the show that the file is unparsable for a time. If I click ''Check Current Change List' again, the check may works then. It seems that the Check is not possible for a while after change file content. (may be until Idea has build index for changed content)
Hmm. I've made no progress on this. The file logic is consistent for the checks - for each file we check if it is either not yet on the disk, or unsaved and, if so, we create a temporary file and scan that. Otherwise we scan using the file on disk. Either way, this logic is run before the scan takes place (in ScannableFile), so there should be no time lag.
Purely speaking in theory it could be related to indexing - we rely on IDEA's API to get the file metadata and decide how to scan it, but it seems odd.
ok I'll try to create a simple test file with small project. I've tested it with default google check rules and still occurs. So it is not an rule config problem. Today I got the error if i click check files for package. And after 20 times the check works. It seems like random.
Maybe I can reproduce it with simple test case
Thank you. I fear my environment may not be a good match for this, as I've only small projects to work with, and an entirely different OS with a different filesystem and approach to file locking.
🤞
Is there anyway to enable debug logging for your plugin to help for looking the cause?
The only way I'm aware of is to clone the project and run the runIdea target - this will start up a version of IDEA 2017 CE, which is the API we compile against, in the debugger and allow you to go mad with breakpoints. It's a bit of a hassle tbh, but very useful.
Thanks I will try it if can reproduce it. But if seem to work with versions before 2019.1.2 Maybe its a version problem but we will see if if reproduce it.
But this will not happen before monday. But then i will try it
Hmm, odd if it does work before 2019.1.2 - there's nothing I'm aware of that should change behaviour in that case.