Import Haskell module or automatically import new Cabal packages

[develop7]

Repro:

1. Have a multi-package Stack project handled with IJ-haskell
2. Create another cabal package inside it and specify it in stack.yaml
3. Open above project in IDEA

Expected: new cabal package gets picked up by IJ-haskell, the way it got picked up when importing project

Instead: it is not. In order to pick up new cabal packages user has to reimport project losing custom modules, local history, configurations etc.

[rikvdkleij]

You can do Tools/Haskell/Update Settings and Restart REPLs

[develop7]

Nope, didn't work for me. With the project I work on, that is.

[rikvdkleij]

Ok, Tools/Haskell/Update Settings and Restart REPLs supports only changes in Cabal file.

[rikvdkleij]

Will fix it when I'm done with getting the UI more responsive/solid.

[rikvdkleij]

Need some time to figure this out.

[rikvdkleij]

Done in beta43. It only adds/imports new modules.

[develop7]

Thanks for implementing this, it does the job in test project indeed, but I've ran into following exception after adding a package to the day job project:

stack trace

java.lang.NullPointerException
	at scala.collection.mutable.ArrayOps$ofRef$.length$extension(ArrayOps.scala:204)
	at scala.collection.mutable.ArrayOps$ofRef.length(ArrayOps.scala:204)
	at scala.collection.IndexedSeqOptimized.segmentLength(IndexedSeqOptimized.scala:196)
	at scala.collection.IndexedSeqOptimized.segmentLength$(IndexedSeqOptimized.scala:195)
	at scala.collection.mutable.ArrayOps$ofRef.segmentLength(ArrayOps.scala:198)
	at scala.collection.GenSeqLike.prefixLength(GenSeqLike.scala:98)
	at scala.collection.GenSeqLike.prefixLength$(GenSeqLike.scala:98)
	at scala.collection.mutable.ArrayOps$ofRef.prefixLength(ArrayOps.scala:198)
	at scala.collection.IndexedSeqOptimized.find(IndexedSeqOptimized.scala:53)
	at scala.collection.IndexedSeqOptimized.find$(IndexedSeqOptimized.scala:52)
	at scala.collection.mutable.ArrayOps$ofRef.find(ArrayOps.scala:198)
	at intellij.haskell.util.HaskellProjectUtil$.findCabalFile(HaskellProjectUtil.scala:147)
	at intellij.haskell.module.HaskellModuleBuilder$.getCabalFile(HaskellModuleBuilder.scala:222)
	at intellij.haskell.module.HaskellModuleBuilder$.createCabalInfo(HaskellModuleBuilder.scala:179)
	at intellij.haskell.module.StackProjectImportBuilder$.$anonfun$addHaskellModule$1(StackProjectImportBuilder.scala:82)
	at intellij.haskell.util.ScalaUtil$$anon$2.compute(ScalaUtil.scala:41)
	at com.intellij.openapi.application.impl.ApplicationImpl.runReadAction(ApplicationImpl.java:963)
	at intellij.haskell.util.ApplicationUtil$.runReadAction(ApplicationUtil.scala:33)
	at intellij.haskell.module.StackProjectImportBuilder$.addHaskellModule(StackProjectImportBuilder.scala:82)
	at intellij.haskell.external.component.StackProjectManager$$anon$2.$anonfun$run$14(StackProjectManager.scala:195)
	at intellij.haskell.external.component.StackProjectManager$$anon$2.$anonfun$run$14$adapted(StackProjectManager.scala:194)
	at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
	at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
	at intellij.haskell.external.component.StackProjectManager$$anon$2.run(StackProjectManager.scala:194)
	at com.intellij.openapi.progress.impl.CoreProgressManager$TaskRunnable.run(CoreProgressManager.java:727)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:164)
	at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:582)
	at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:532)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:87)
	at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:151)
	at com.intellij.openapi.progress.impl.CoreProgressManager$4.run(CoreProgressManager.java:403)
	at com.intellij.openapi.application.impl.ApplicationImpl$1.run(ApplicationImpl.java:314)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

The stack trace looks very relevant to this issue, unfortunately I'm not sure what I was doing wrong.

[rikvdkleij]

Thanks for reporting!

Used this feature already couple of time within large projects and worked as expected so must be some corner case.

Looking to the stacktrace it could be that no Cabal file was found but library code should not crash....

[rikvdkleij]

Can you display the directory listing of the root folder of the package you wanted to add?

[develop7]

All right, here's a redacted screenshot. The package directory name, .cabal file name and package name in the .cabal file are all the same.

[rikvdkleij]

Thanks!

No hidden files?

[develop7]

none, besides usual .gitignore & .stack-work

BTW copying&renaming .iml file from sibling package & referencing it in .idea/modules.xml did the job

[rikvdkleij]

BTW copying&renaming .iml file from sibling package & referencing it in .idea/modules.xml did the job

Ok, but that is just hacking :smile:

Like I said, importing new modules in existing projects worked for me. I'm curious why it does not work in your case.

[rikvdkleij]

Other question :smile:

How do you refer in the stack.yaml to that other project/module?

[develop7]

by simply adding the

- REDACTED_PACKAGE_NAME

line under packages key

[rikvdkleij]

So not using a relative or absolute file path?

[develop7]

I mean, it is relative already, to the stack project root.

[rikvdkleij]

Okay, I get exactly the same error (stacktrace) when there exists a non-existing file path in packages.

Only thing the plugin can do is check if file path exists before adding new module.

[rikvdkleij]

I have tried to reproduce your issue with new package to import in project folder but without success.

[rikvdkleij]

I found another cause of this error. When there are git(hub) references defined in packages.

[develop7]

Yes there are in my stack.yaml! They should've been added as modules too, but there's no way to ask stack to list directories with packages' source code.

[rikvdkleij]

They should've been added as modules too, but there's no way to ask stack to list directories with packages source code.

Well, I could download code and add them as modules but that is some work and have to do some more investigation. Also, if github reference is changed, plugin has to update the code in some way. Does package have to be seen as part of project or as external library?

[develop7]

To think of it, all of the VCS "packages" actually had an extra-dep: true stanza, which means they must end up on the extra-deps section (the extra-dep: true isn't mentioned in Stack docs at the moment). Therefore I should these "packages" to extra-deps first and the issue would just go away.

Well, I could download code and add them as modules but that is some work and have to do some more investigation.

Stack checks out VCS/http dependencies under .stack-work/downloaded/<seemingly_random_name>/, so the question is to actually figure out which dependency ends what directory. In fact this is the exactly the way extra-deps are handled too.

Also, if github reference is changed, plugin has to update the code in some way.

No need, stack build got it covered. I mean all you have to update is the module source directory, because if reference gets changed, the path under .stack-work/downloaded is changed too.

Does package have to be seen as part of project or as external library?

Actually I'd go with external library unconditionally, at least at first as lower-hanging fruit. Also because the source ends up in .stack-work/downloaded/<random_name> anyway, tracking which only complicates things.

[rikvdkleij]

@develop7 Is this issue solved?

[develop7]

Actually it's still there. Just recently (May 2 to be exact) I've hacked .idea/modules.xml so the new subpackage would be picked up.

[rikvdkleij]

Do you have an example project so I can try to reproduce it? Must be something unexpected in the stack.yaml.

[develop7]

Not really, it was that huge project on the job. Will try to reproduce the issue against the sample project bit later.

[rikvdkleij]

@develop7 ping :smile: