KT-47511: Incremental build compatibility with gradle composing builds

[morefreefg]

Kotlin-gradle-plugin maintains the dependency relationship between modules from gradle taskGraph, and it is used in incremental compilation.

Assume there are two modules called moduleA and moduleB. If modulesA depends on moduleB, during once compilation, moduleB changed, moduleA will find out the 'build' directory of moduleB to determine if moduleA should be re-compiled.

But when moduleB is a composing build, incremental compilation will fail because moduleA have no idea about what moduleB is. when moduleB changes, moduleA will always perform full compilation.

It's a bug in kotlin-gradle-plugin, kotlin didn't take this situation into account

To briefly illustrate this case, here is a demo-application project attached.

1. run ./gradlew assembleDebug
2. modify included/lib/src/main/java/me/fengguang/lib/KTClassInIncludedLib.kt, for example: Append strings at line 8.
3. run ./gradlew assembleDebug --debug | grep -e ".*\[KOTLIN\] \[IC\] Non-incremental compilation will be performed.*"

You will see, there is a debug message printed at the console. 2021-06-30T10:58:17.992+0800 [DEBUG] [org.gradle.api.Task] [KOTLIN] [IC] Non-incremental compilation will be performed: UNKNOWN_CHANGES_IN_GRADLE_INPUTS

This log indicated that moduleA(:app in demoapplication) performs re-compile because classes.jar in moduleB(:lib in demo-application) changed, and moduleA cannot analyze what's going on with this jar file because moduleB didn't show up at moduleA's dependency graph.

[morefreefg]

Any update on this?

[LifeIsStrange]

@morefreefg friendly ping :)

[franckrasolo]

The following snippet is from a Kotlin build report for a Gradle multi-project with a composite build.

It appears to confirm this bug on successive invocations of the same Gradle task:

Gradle start parameters:
  tasks = [:foo:bar:test]
  excluded tasks = []
  current dir = /builds/my/project
  project properties args = {}
  system properties args = {}

Time metrics:
  Total Gradle task time: 161.79 s
  Task action: 0.44 s
    Clear output: 0.00 s
    Connect to Kotlin daemon: 1.59 s
    Clear jar cache: 0.03 s
    Calculate output size: 0.02 s
    Run compilation: 130.20 s
      Incremental compilation in daemon: 128.32 s
        Store build info: 0.00 s
        Clear outputs on rebuild: 0.01 s
        Update caches: 1.61 s
        Sources compilation round: 123.37 s
        Write history file: 0.00 s
        Shrink and save current classpath snapshot after compilation: 1.57 s
          Shrink current classpath snapshot non-incrementally: 1.53 s
            Load current classpath snapshot: 0.59 s
          Save shrunk current classpath snapshot: 0.03 s
  Compiler initialization time: 3.97 s
  Compiler code analysis: 83.29 s
  Compiler code generation: 28.22 s

Size metrics:
  Total size of the cache directory: 6.0 MB
    ABI snapshot size: 245 B
  Total compiler iteration: 5
  Number of times classpath snapshot is shrunk and saved after compilation: 5
    Number of classpath entries: 567
    Size of classpath snapshot: 48.6 MB
    Size of shrunk classpath snapshot: 878.1 KB
  Number of cache misses when loading classpath entry snapshots: 152

Build attributes:
  REBUILD_REASON: Unknown Gradle changes

Total time for Kotlin tasks: 137.06 s (84.7 % of all tasks time)

 Time   | % of Kotlin time | Task
43.19 s | 31.5 %           | :foo:infra-testing:compileTestFixturesKotlin
42.10 s | 30.7 %           | :foo:infra:compileKotlin
33.37 s | 24.3 %           | :foo:bar:compileKotlin
16.90 s | 12.3 %           | :foo:bar:compileTestKotlin
 1.51 s |  1.1 %           | :foo:bar:compileTestFixturesKotlin