Java/Scala: Improve resources handling

[lefou]

Mill currently only has one resources target. It's content will end up in the built jar and also in the classpath. But it is also evaluate before and feed into compile target. This has various implications:

• Each change to resources will trigger a recompilation
• Generated resources (after compile) are not supported

See also the following discussion on Gitter: https://gitter.im/lihaoyi/mill?at=6246909d9a09ab24f3d00982

Victor Borja @vic 07:41 Hello, running compile on a ~400 scala source files module I've noticed something weird: turns out if you do mill inspect foo.compileClasspath it includes the output of foo.resources that is somewhat surprising to me, and noticed because if I change eg, foo/resources/application.conf then that will cause scalac to compile all ~400 sources again, just for changing a value that will be used at runtime. So in my project I fixed it by overriding the compileClasspath task and removing the same module resources from it. I believe if some project compilation depends on it's own resources, they should be explicitly added to compileClasspath instead of it being the default. The only way I imagine it useful for resources being on compileClasspath of the SAME module would be for some advanced macros to generate code from resources or something. On the other hand, editing a foo/resources/application.conf or whatever is a pretty much more common scenario and that would likely trigger compilation since one of the compile task input changes. I'd be happy to provide a pull-request if you people think this is something worth fixing.

Tobias Roeser @lefou 08:39 @vic In the past, I though about adding some "runtimeResources" multiple times. Something that ends up in the final JAR and/or the runtime classpath. The motivation was to add stuff that gets generated after the compilation. It's probably the same feature but motivated from the other side. As we probably going to need both: resources in the compile classpath and resources after, we may want to avoid breaking compatibility and just add a new runResources? WDYT?

Andrew Richards @ajrnz 10:02 Regarding the resources issue I'd vote for changing the default. I come across this issue when modifying application.conf, logback.xml, web content and probably more file types. Apart from the macros example above I can't think of any reason to have them on the compile path (counter examples welcome). In the past I've removed the resources from the compile path when doing for example web development where the .html, .css etc were in resources. Adding a runResources would require a new directory and a name for it and would be doing what I imagine most people want by default.

To improve the resource handling, I'm interested in your opinion. Which route do you prefer?

1. Break behavioral compatibility: No longer feed resources into compile and add a compileResource target which is feed into compile instead
2. Keep behavior and the resources target as is and introduce a new runResources

[lihaoyi]

I'm in favor of (1.).

I'm having trouble coming up with an example where having resources during compilation is necessary. I know in theory macros or compile plugins could read resources, but I don't know of any existing macros or plugins that actually do.

Combining the relativey low chance of semantic breakage, with the fact that this is a binary compatible change, makes me feel like accepting the small semantic breakage will be less painful than having to for-ever-after tell people "use runResources, it's what you actually want, not resources"

[lefou]

We may also collect some info how other build tools handle resources, e.g. sbt, Maven, Gradle, etc. Despite of our decision, when it is significantly different than in other tools, we may want to add some clarifying text to the documentation.

[lefou]

If we decide to go with a semantic change, we should still increment to the next Mill binary platform version, as external modules will need a way to deal with the changed semantics. E.g. my plugins which touch the compiler or packaging aspects (mill-kotlin, mill-aspectj, mill-osgi) will go crazy, if we introduce this change in a patch level release.

[vic]

I'd be interested in implementing this (1.), of course I'll need a lot of direction along the way and will ask for reviews/comments but feel free to assign this to me :)

[lefou]

I'd be interested in implementing this (1.), of course I'll need a lot of direction along the way and will ask for reviews/comments but feel free to assign this to me :)

Great! Just open a draft pull request once you have some code.