Add incremental compilation to kotlinc

[cbeust]

Recommendations from Ilya Chernikov:

The best source is probably the one in the gradle plugin, in particular

org.jetbrains.kotlin.gradle.tasks.KotlinCompile.callCompiler

Exact link: https://github.com/JetBrains/kotlin/blob/master/libraries/tools/kotlin-gradle-plugin-core/src/main/kotlin/org/jetbrains/kotlin/gradle/tasks/Tasks.kt#L139

It basically works like this: in the incremental compilation cache we store the so-called “lookups”, the references from compiled objects to other symbols, which we collect on the previous compilations. Now wen we are compiling first bunch of sources (changed files), we collect compiled symbols from them, filter changed ones since last compilation and look into lookup table to see if there are other objects that need to be recompiled because of that.

On top of that there is some more logic on failed previous compilation, and may be some other stuff. You actually may try to reuse the logic from gradle task almost 1:1, just drop gradle specific stuff.

As far as I see it is not gradle-specific, except may be minor details. It contains a logic of incremental/non-incremental decision, calculation of affected sources list, and incremental compilation loop.

The one in IDEA build tool probably will be much more difficult to grasp, because it relies on some logic and infrastructure of the build tool. But you may have a look around org.jetbrains.kotlin.jps.build.KotlinBuilder class

The common utilities for both parts is extracted into a build-common lib - org.jetbrains.kotlin.incremental - etc.

alexey.tsvetkov knows more.