WIP: Allow some task reference from source tasks
TBD
With the merge of https://github.com/com-lihaoyi/mill/pull/4524, source tasks could no longer depend on other tasks. Due to this change, the logic to calculate changes, especially in the --watch mode complicated, could be significantly simplified. Unfortunately, this also broke a common use-case, where sub-modules or cross-modules want to derive there source paths from the outer or parent module to provide a convenient and consistent default behavior. for that reason, the def sourceFolders: Seq[os.SubPath] was introduced, which could be modified in an object-orients fashion to cusomites the default behavior of sources. Although looking simple on paper, when facing an existing grown setup, it makes it really hard to understand the effective source path just from looking at the build script and modules code.
Here are some issues I noticed:
sourcesandsourceFolderscan be independently overridden without any consistency guarantees. It's unclear at a concrete module, mixing in various traits what their actual value issourceFoldersisn't a task, hence it can't be easily shown and inspected withmill show.- Overriding
sourcesmay subsequently ignore or re-use the value ofsourceFolders, leaving the maintainer in doubt about the actual paths used - The existing module structure does not use
sourceFolders. In fact, existing source dir customization (e.g. in the Android and Kotlin modules) complete bypassessourceFoldersand just overridesources, so the refactoring towardssourceFoldersis incomplete - Manually understanding of the actual value of
sourcesis a tedious and error-prone process. I tried to understand the value ofAndroidKotlinAppModule.sourcesin a PR, where I neither could just runmill shownor could I simply wald thesuper.sourcestree up. It costs a lot of time to get an understanding of such simple core setting of a module, leaving too little confidence in a setup involving lots of mix-ins.
Consequences:
I think have two parallel structures the configure the sources (sources and sourceFolders) is not going to age well. I already find it difficult to work with it. As a maintainer of various larger polyglot projects this change significantly raises the mental overhead. Therefore I think we should try to handle Task.Sources derivations not outside of it. Instead we should allow task dependencies as we did before and remove sourceFolders. Each module level can be inspected, the effective values can be shown. There is no need to use println-debugging or a JVM debugger to understand the effective values of sourceFolders.
What this POC tries to change:
Re-enable the ability to let Task.Sources depend on other task, but limit the allowed task types to just Sources tasks or maybe tasks returning the type Seq[PathRef] (and compatible). That way, the logic to re-evaluate the up-to-date state should be kept simple. We keep just checking the PathRefs the final sources task is returning for changes.