NullPointerException in MigrationScriptReaderImpl

[felixkrull-neuland]

When running tests that internally use ElasticsearchEvolutions (to set up indices for an integration test) with sbt, I get a NullPointerException while loading the migration scripts:

[info]     java.lang.NullPointerException:
[info]     at java.base/java.io.Reader.<init>(Reader.java:167)
[info]     at java.base/java.io.InputStreamReader.<init>(InputStreamReader.java:113)
[info]     at com.senacor.elasticsearch.evolution.core.internal.migration.input.MigrationScriptReaderImpl.lambda$readScriptsFromClassPath$3(MigrationScriptReaderImpl.java:154)
[info]     at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271)
[info]     at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
[info]     at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
...

To make matters more fun, this only happens when running all our tests, not when running just the evolution-using test. I can try to create a minimal example if necessary, but this complication makes that rather tricky.

What I Know Though

I narrowed the problem to sbt doing weird classloader things. There's multiple aspects that I can find:

1. ClassLoader.getResourceAsStream returns null if it can't find or can't open the resource. Having a nicer exception for that case would make sense.
2. Different class loaders are used for enumerating and for loading scripts. ClasspathHelper.forPackage uses both the thread-context ClassLoader and its own ClassLoader to enumerate scripts, but loading happens via the thread class loader. That's why it happily finds the scripts, but can't open them:

17:15:45.050 [info] - [ElasticsearchEvolution.java:102] - reading migration scripts...
17:15:45.070 [debug] - [MigrationScriptReaderImpl.java:153] - reading migration script 'evolutions/opensearch/V1.0__script.http' from classpath...
17:15:45.070 [trace] - [MigrationScriptReaderImpl.java:191] - getDefaultClassLoader - Thread.currentThread().getContextClassLoader()='sbt.internal.LayeredClassLoader@592e586d'

3. In this specific configuration (sbt with multiple in-tree modules), the thread class loader apparently can't load these resource files. Which sucks and is just one more reason for me to dislike sbt, but what can you do.

I don't know what the solution for point 3 is. Would it be possible to load the scripts using different class loaders much as it is already possible to enumerate scripts across class loaders?

[xtermi2]

You are talking about sbt. This means you are usung elasticsearch-evolution in a Scala project, right? Have you found any solution so far? I think I can't help you at this stage. Can you provide a self contained demo project to reproduce this issue?

[felixkrull-neuland]

Yeah, this is a Scala project. The reference to an sbt ClassLoader suggests it's something sbt does though, rather than Scala or Scalatest. I've managed to work around it for now by using the file path in the tests and the resources path for deployment. It's not terribly elegant, but it works for now.

I'll try and create a minimal example. It might be a while though.