Perl5-IDEA icon indicating copy to clipboard operation
Perl5-IDEA copied to clipboard
verified publisher icon Camelcade

Do not use stubs while building stubs

Open hurricup opened this issue 4 years ago • 1 comments 

Build: IU-203.7717.56
OS: Linux amd64 5.8.5-050805-generic
Java Vendor: JetBrains s.r.o.
Java Version: 11.0.10
Java Arch: 64 bit
Java Runtime Version: 11.0.10+8-b1145.96
Perl Plugin Version: 203.5784.10-EAP-SNAPSHOT
Description: <none>
Message: none
Throwable:
java.lang.Throwable: Indexing process should not rely on non-indexed file data.
Indexing file = /home/cam/dev/gitlab.com/camstuartdev/act/app/script/backtest/strategy-crius, file id = 27274, queried file id = 12777
	at com.intellij.openapi.diagnostic.Logger.error(Logger.java:165)
	at com.intellij.indexing.shared.platform.impl.MergedInvertedIndex.getIndexedFileData(MergedInvertedIndex.java:206)
	at com.intellij.util.indexing.FileBasedIndexEx.lambda$getFileData$3(FileBasedIndexEx.java:168)
	at com.intellij.util.indexing.FileBasedIndexEx.lambda$processExceptions$5(FileBasedIndexEx.java:234)
	at com.intellij.util.ConcurrencyUtil.withLock(ConcurrencyUtil.java:229)
	at com.intellij.util.indexing.FileBasedIndexEx.lambda$processExceptions$6(FileBasedIndexEx.java:234)
	at com.intellij.util.indexing.IndexAccessValidator.validate(IndexAccessValidator.java:33)
	at com.intellij.util.indexing.FileBasedIndexEx.processExceptions(FileBasedIndexEx.java:234)
	at com.intellij.util.indexing.FileBasedIndexEx.getFileData(FileBasedIndexEx.java:161)
	at com.intellij.util.indexing.FileBasedIndexImpl.getFileData(FileBasedIndexImpl.java:1051)
	at com.intellij.psi.stubs.StubTreeLoaderImpl.readFromVFile(StubTreeLoaderImpl.java:99)
	at com.intellij.psi.stubs.StubTreeLoaderImpl.readOrBuild(StubTreeLoaderImpl.java:42)
	at com.intellij.psi.impl.source.PsiFileImpl.getStubTree(PsiFileImpl.java:630)
	at com.intellij.psi.impl.source.PsiFileImpl.getGreenStubTree(PsiFileImpl.java:951)
	at com.intellij.psi.impl.source.PsiFileImpl.getStubbedSpine(PsiFileImpl.java:256)
	at com.intellij.psi.stubs.StubProcessingHelperBase.lambda$getAllSpines$0(StubProcessingHelperBase.java:84)
	at com.intellij.util.containers.ContainerUtil.map(ContainerUtil.java:1983)
	at com.intellij.psi.stubs.StubProcessingHelperBase.getAllSpines(StubProcessingHelperBase.java:84)
	at com.intellij.psi.stubs.StubProcessingHelperBase.processStubsInFile(StubProcessingHelperBase.java:64)
	at com.intellij.psi.stubs.StubIndexImpl.lambda$processElements$2(StubIndexImpl.java:291)
	at com.intellij.psi.stubs.StubIndexImpl.processElements(StubIndexImpl.java:327)
	at com.intellij.psi.stubs.StubIndex.processElements(StubIndex.java:49)
	at com.perl5.lang.perl.psi.stubs.PerlStubIndexBase.processElements(PerlStubIndexBase.java:61)
	at com.perl5.lang.perl.util.PerlPackageUtil.processNamespaces(PerlPackageUtil.java:362)
	at com.perl5.lang.perl.util.PerlPackageUtil.getNamespaceDefinitions(PerlPackageUtil.java:328)
	at com.perl5.lang.perl.psi.mro.PerlMro.getPackageParents(PerlMro.java:185)
	at com.perl5.lang.perl.psi.mro.PerlMro.getLinearISA(PerlMro.java:169)
	at com.perl5.lang.perl.idea.codeInsight.typeInference.value.PerlCallObjectValue.processCallTargets(PerlCallObjectValue.java:120)
	at com.perl5.lang.perl.idea.codeInsight.typeInference.value.PerlCallValue.processCallTargets(PerlCallValue.java:163)
	at com.perl5.lang.perl.extensions.mojo.MojoLightDelegatingSubDefinition.processTargets(MojoLightDelegatingSubDefinition.java:124)
	at com.perl5.lang.perl.extensions.mojo.MojoLightDelegatingSubDefinition.getSubArgumentsList(MojoLightDelegatingSubDefinition.java:55)
	at com.perl5.lang.perl.psi.light.PerlLightSubDefinitionElement.equals(PerlLightSubDefinitionElement.java:141)
	at com.intellij.util.IdempotenceChecker.checkPsiEquivalence(IdempotenceChecker.java:267)
	at com.intellij.util.IdempotenceChecker.checkValueEquivalence(IdempotenceChecker.java:182)
	at com.intellij.util.IdempotenceChecker.checkArrayEquivalence(IdempotenceChecker.java:305)
	at com.intellij.util.IdempotenceChecker.checkCollectionElements(IdempotenceChecker.java:285)
	at com.intellij.util.IdempotenceChecker.checkValueEquivalence(IdempotenceChecker.java:167)
	at com.intellij.util.IdempotenceChecker.checkCachedValueData(IdempotenceChecker.java:232)
	at com.intellij.util.IdempotenceChecker.checkValueEquivalence(IdempotenceChecker.java:164)
	at com.intellij.util.IdempotenceChecker.checkEquivalence(IdempotenceChecker.java:73)
	at com.intellij.util.CachedValueBase.getValueWithLock(CachedValueBase.java:245)
	at com.intellij.psi.impl.PsiCachedValueImpl.getValue(PsiCachedValueImpl.java:43)
	at com.intellij.util.CachedValuesManagerImpl.getCachedValue(CachedValuesManagerImpl.java:78)
	at com.intellij.psi.util.CachedValuesManager.getCachedValue(CachedValuesManager.java:150)
	at com.intellij.psi.util.CachedValuesManager.getCachedValue(CachedValuesManager.java:120)
	at com.perl5.lang.perl.psi.impl.PerlPolyNamedElement.getLightElements(PerlPolyNamedElement.java:56)
	at com.perl5.lang.perl.psi.stubs.PerlPolyNamedElementType.createStub(PerlPolyNamedElementType.java:66)
	at com.perl5.lang.perl.psi.stubs.PerlPolyNamedElementType.createStub(PerlPolyNamedElementType.java:37)
	at com.intellij.psi.stubs.DefaultStubBuilder$StubBuildingWalkingVisitor.createStub(DefaultStubBuilder.java:85)
	at com.intellij.psi.stubs.DefaultStubBuilder$StubBuildingWalkingVisitor.visitNode(DefaultStubBuilder.java:60)
	at com.intellij.psi.stubs.DefaultStubBuilder$StubBuildingWalkingVisitor.buildStubTree(DefaultStubBuilder.java:55)
	at com.intellij.psi.stubs.DefaultStubBuilder.buildStubTreeFor(DefaultStubBuilder.java:33)
	at com.intellij.psi.stubs.DefaultStubBuilder.buildStubTree(DefaultStubBuilder.java:22)
	at com.intellij.psi.stubs.StubTreeBuilder.buildStubTree(StubTreeBuilder.java:122)
	at com.intellij.psi.stubs.StubUpdatingIndex$1.computeValue(StubUpdatingIndex.java:172)
	at com.intellij.psi.stubs.StubUpdatingIndex$1.computeValue(StubUpdatingIndex.java:150)
	at com.intellij.psi.stubs.StubUpdatingIndex$1.computeValue(StubUpdatingIndex.java:117)
	at com.intellij.util.indexing.SingleEntryIndexer.map(SingleEntryIndexer.java:30)
	at com.intellij.util.indexing.SingleEntryIndexer.map(SingleEntryIndexer.java:19)
	at com.intellij.util.indexing.impl.MapReduceIndex.mapByIndexer(MapReduceIndex.java:291)
	at com.intellij.util.indexing.impl.MapReduceIndex.mapInput(MapReduceIndex.java:283)
	at com.intellij.util.indexing.impl.storage.VfsAwareMapReduceIndex.mapInput(VfsAwareMapReduceIndex.java:168)
	at com.intellij.util.indexing.impl.storage.VfsAwareMapReduceIndex.mapInput(VfsAwareMapReduceIndex.java:46)
	at com.intellij.util.indexing.impl.MapReduceIndex.mapInputAndPrepareUpdate(MapReduceIndex.java:226)
	at com.intellij.psi.stubs.StubUpdatingIndex$MyIndex.mapInputAndPrepareUpdate(StubUpdatingIndex.java:448)
	at com.intellij.psi.stubs.StubUpdatingIndex$MyIndex.mapInputAndPrepareUpdate(StubUpdatingIndex.java:408)
	at com.intellij.indexing.shared.platform.impl.MergedInvertedIndex.updateBaseIndex(MergedInvertedIndex.java:330)
	at com.intellij.indexing.shared.platform.impl.MergedInvertedIndex.mapInputAndPrepareUpdate(MergedInvertedIndex.java:116)
	at com.intellij.indexing.shared.platform.impl.MergedInvertedIndex.mapInputAndPrepareUpdate(MergedInvertedIndex.java:38)
	at com.intellij.util.indexing.FileBasedIndexImpl.lambda$indexUnsavedDocument$12(FileBasedIndexImpl.java:1027)
	at com.intellij.openapi.fileTypes.impl.FileTypeManagerImpl.freezeFileTypeTemporarilyIn(FileTypeManagerImpl.java:555)
	at com.intellij.util.indexing.FileBasedIndexImpl.indexUnsavedDocument(FileBasedIndexImpl.java:999)
	at com.intellij.util.indexing.RegisteredIndexes$DocumentUpdateTask.doProcess(RegisteredIndexes.java:181)
	at com.intellij.util.indexing.RegisteredIndexes$DocumentUpdateTask.doProcess(RegisteredIndexes.java:172)
	at com.intellij.util.indexing.UpdateTask.process(UpdateTask.java:64)
	at com.intellij.util.indexing.UpdateTask.processAll(UpdateTask.java:33)
	at com.intellij.util.indexing.FileBasedIndexImpl.lambda$indexUnsavedDocuments$11(FileBasedIndexImpl.java:955)
	at com.intellij.util.indexing.StorageBufferingHandler.runUpdate(StorageBufferingHandler.java:33)
	at com.intellij.util.indexing.FileBasedIndexImpl.indexUnsavedDocuments(FileBasedIndexImpl.java:955)
	at com.intellij.util.indexing.FileBasedIndexImpl.ensureUpToDate(FileBasedIndexImpl.java:730)
	at com.intellij.psi.stubs.StubIndexImpl.getContainingIds(StubIndexImpl.java:465)
	at com.intellij.psi.stubs.StubIndexImpl.processElements(StubIndexImpl.java:298)
	at com.intellij.psi.stubs.StubIndex.getElements(StubIndex.java:105)
	at com.intellij.psi.stubs.StubIndex.getElements(StubIndex.java:93)
	at org.jetbrains.kotlin.idea.stubindex.KotlinFileFacadeFqNameIndex.get(KotlinFileFacadeFqNameIndex.kt:19)
	at org.jetbrains.kotlin.idea.caches.resolve.IDEKotlinAsJavaSupport$findFilesForFacade$1.invoke(IDEKotlinAsJavaSupport.kt:249)
	at org.jetbrains.kotlin.idea.caches.resolve.IDEKotlinAsJavaSupport$findFilesForFacade$1.invoke(IDEKotlinAsJavaSupport.kt:39)
	at org.jetbrains.kotlin.idea.util.application.ApplicationUtilsKt$sam$com_intellij_openapi_util_Computable$0.compute(ApplicationUtils.kt)
	at com.intellij.openapi.application.impl.ApplicationImpl.runReadAction(ApplicationImpl.java:871)
	at org.jetbrains.kotlin.idea.util.application.ApplicationUtilsKt.runReadAction(ApplicationUtils.kt:17)
	at org.jetbrains.kotlin.idea.caches.resolve.IDEKotlinAsJavaSupport.findFilesForFacade(IDEKotlinAsJavaSupport.kt:248)
	at org.jetbrains.kotlin.idea.caches.resolve.IDEKotlinAsJavaSupport.getFacadeClasses(IDEKotlinAsJavaSupport.kt:181)
	at org.jetbrains.kotlin.asJava.finder.JavaElementFinder.findClasses(JavaElementFinder.kt:58)
	at org.jetbrains.kotlin.asJava.finder.JavaElementFinder.findClass(JavaElementFinder.kt:46)
	at com.intellij.psi.impl.JavaPsiFacadeImpl.doFindClass(JavaPsiFacadeImpl.java:89)
	at com.intellij.psi.impl.JavaPsiFacadeImpl.findClass(JavaPsiFacadeImpl.java:67)
	at com.intellij.jam.JavaLibraryUtils.lambda$getLibraryClassMap$0(JavaLibraryUtils.java:46)
	at com.intellij.util.containers.ConcurrentFactoryMap$2.create(ConcurrentFactoryMap.java:174)
	at com.intellij.util.containers.ConcurrentFactoryMap.get(ConcurrentFactoryMap.java:40)
	at java.base/java.util.concurrent.ConcurrentMap.getOrDefault(ConcurrentMap.java:88)
	at com.intellij.jam.JavaLibraryUtils.hasLibraryClass(JavaLibraryUtils.java:29)
	at com.intellij.quarkus.QuarkusUtils.hasQuarkusLibrary(QuarkusUtils.kt:17)
	at com.intellij.quarkus.config.QsConfigKeyReferenceSearcher.processQuery(QsConfigKeyReferenceSearcher.kt:20)
	at com.intellij.quarkus.config.QsConfigKeyReferenceSearcher.processQuery(QsConfigKeyReferenceSearcher.kt:18)
	at com.intellij.openapi.application.QueryExecutorBase.execute(QueryExecutorBase.java:77)
	at com.intellij.util.ExecutorsQuery.processResults(ExecutorsQuery.java:28)
	at com.intellij.util.AbstractQuery.doProcessResults(AbstractQuery.java:99)
	at com.intellij.util.AbstractQuery.delegateProcessResults(AbstractQuery.java:116)
	at com.intellij.util.MergeQuery.processResults(MergeQuery.java:22)
	at com.intellij.util.AbstractQuery.doProcessResults(AbstractQuery.java:99)
	at com.intellij.util.AbstractQuery.delegateProcessResults(AbstractQuery.java:116)
	at com.intellij.util.UniqueResultsQuery.processResults(UniqueResultsQuery.java:39)
	at com.intellij.util.AbstractQuery.doProcessResults(AbstractQuery.java:99)
	at com.intellij.util.AbstractQuery.forEach(AbstractQuery.java:91)
	at com.intellij.util.AbstractQuery.findAll(AbstractQuery.java:28)
	at com.intellij.util.AbstractQuery.iterator(AbstractQuery.java:39)
	at com.perl5.lang.perl.idea.intellilang.PerlStringLanguageInjector.computeInjectionContext(PerlStringLanguageInjector.java:131)
	at com.perl5.lang.perl.idea.intellilang.PerlStringLanguageInjector.lambda$getPerlInjectionContext$0(PerlStringLanguageInjector.java:89)
	at com.intellij.openapi.util.RecursionManager$1.computePreventingRecursion(RecursionManager.java:111)
	at com.intellij.openapi.util.RecursionGuard.doPreventingRecursion(RecursionGuard.java:42)
	at com.intellij.openapi.util.RecursionManager.doPreventingRecursion(RecursionManager.java:68)
	at com.perl5.lang.perl.idea.intellilang.PerlStringLanguageInjector.getPerlInjectionContext(PerlStringLanguageInjector.java:88)
	at com.perl5.lang.perl.idea.intellilang.PerlStringLanguageInjector.getLanguagesToInject(PerlStringLanguageInjector.java:67)
	at com.intellij.psi.impl.source.tree.injected.InjectedLanguageManagerImpl.processInPlaceInjectorsFor(InjectedLanguageManagerImpl.java:418)
	at com.intellij.psi.impl.source.tree.injected.InjectedLanguageUtilBase.probeElementsUp(InjectedLanguageUtilBase.java:207)
	at com.intellij.psi.impl.source.tree.injected.InjectedLanguageUtilBase.enumerate(InjectedLanguageUtilBase.java:137)
	at com.intellij.psi.impl.source.tree.injected.InjectedLanguageManagerImpl.enumerateEx(InjectedLanguageManagerImpl.java:318)
	at com.intellij.codeInsight.daemon.impl.LineMarkersPass.queryLineMarkersForInjected(LineMarkersPass.java:211)
	at com.intellij.codeInsight.daemon.impl.LineMarkersPass.queryProviders(LineMarkersPass.java:173)
	at com.intellij.codeInsight.daemon.impl.LineMarkersPass.lambda$doCollectInformation$3(LineMarkersPass.java:91)
	at com.intellij.codeInsight.daemon.impl.Divider.divideInsideAndOutsideInOneRoot(Divider.java:81)
	at com.intellij.codeInsight.daemon.impl.LineMarkersPass.doCollectInformation(LineMarkersPass.java:78)
	at com.intellij.codeHighlighting.TextEditorHighlightingPass.collectInformation(TextEditorHighlightingPass.java:56)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$doRun$1(PassExecutorService.java:400)
	at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1137)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$doRun$2(PassExecutorService.java:393)
	at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:658)
	at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:610)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:65)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.doRun(PassExecutorService.java:392)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$run$0(PassExecutorService.java:368)
	at com.intellij.openapi.application.impl.ReadMostlyRWLock.executeByImpatientReader(ReadMostlyRWLock.java:172)
	at com.intellij.openapi.application.impl.ApplicationImpl.executeByImpatientReader(ApplicationImpl.java:183)
	at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.run(PassExecutorService.java:366)
	at com.intellij.concurrency.JobLauncherImpl$VoidForkJoinTask$1.exec(JobLauncherImpl.java:188)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

hurricup avatar Apr 02 '21 11:04 hurricup

Another root issue: https://camelcade.myjetbrains.com/youtrack/issue/CAMELCADE-18243/java.lang.Throwable-Indexing-process-should-not-rely-on-non-indexed-file-data.

hurricup avatar Mar 29 '24 11:03 hurricup

So, what is going on here. When building stubs for the light elements, we are using CVM which may perform random idempotency checking. And it involves equality checking for the light elements we collected.

For some reason, mojo lite methods were implemented via runtime declarations (acutally perl does this, so I probably just did the same). Atm for mojo lite we are not just exporting some implicitly declared methods, but creating some light declarations for each file with mojo lite usage. And they are delegating arguments computation to their counterparts. And comparison of them require indexes usage.

Feels like to fix this i need to change mojo lite approach to the implicit declarations + imports. And this should work just fine.

hurricup avatar Sep 07 '24 14:09 hurricup