[work in progress] Coroutines the Third

[Aidan63]

Third times the charm, right?

This is an evolution of #11554, the core state machine transformation has remained pretty much untouched from that branch, but the surrounding scaffolding code has been changed to more closely align with kotlin's coroutine implementation. This should hopefully address the issues surrounding continuation, scheduling, and a few others which were discovered in the other PR.

Changes

Coroutine functions are now transformed to accept a Continuation<Any> argument, return Any, and contain the state machine transformation. The continuation argument either holds the state of a paused execution of the coroutine function, or what will be invoked once the coroutine completes. The any return will either be the result of the coroutines execution, or a special token indicating that it's execution has been suspended.

@:coroutine function foo(i:Int):String {}

function foo(i:Int, _hx_continuation:ICoroutine<Any>):Any {}

In addition each coroutine function has a class generated for it. This class implements IContinuation<Any> and stores data related to the execution of a coroutine, this includes, the current state, the result or error, variables accessed across states, and a reference to the continuation which will be invoked once this coroutine completes, essentially creating a chain. The continuation interface contains a resume function which when called resumes execution of the coroutine and is responsible to scheduling the execution of the coroutine into an appropriate thread.

class HxCoro__Main_foo implements IContinuation<Any> {
	public var _hx_result : Any;

	public var _hx_error : Exception;

	public var _hx_state : Int;

	public var _hx_context : CoroutineContext;

	public var _hx_completion : IContinuation<Any>;

	public function resume(result:Any, error:Exception) {
		_hx_result = result;
		_hx_error  = error;

		_hx_context.scheduler.schedule(() -> {
			try {
				var result = Main.foo(0, this);
				if (result is Primitive) {
					return;
				} else {
					_hx_completion.resume(result, null);
				}
			} catch (exn:Exception) {
				_hx_completion.resume(null, exn);
			}
		});
	}
}

Coroutines can be started with the blocking function Coroutine.run, which blocks until a coroutine completes. Internally it has its own event loop which all coroutine continuations are scheduled onto.

A quick hello world example is provided below. It should work on all sys targets, but some targets might need their generators updated.

import haxe.coro.Coroutine;
import haxe.coro.Coroutine.delay;

@:coroutine function test() {
    fancyPrint('hello');

    delay(1000);

    fancyPrint('world');
}

@:coroutine function fancyPrint(text:String):Void {
    for (i in 0...text.length) {
        Sys.print(text.charAt(i));

        delay(100);
    }

    Sys.print('\n');
}

function main() {
    Coroutine.run(test);
}

Resources

I'm putting a few links below, specifically related to kotlin's coroutines, which helped the puzzle come together a bit for me.

https://kt.academy/article/cc-under-the-hood

That is a good overview of the underlying machinery of kotlin's coroutines. Jumping straight into decompiled kotlin is very daunting as there are many, many layers of abstraction due to optimisations, features, etc. That article cuts through all of it and gets right to the core of whats happening. I pretty much copied that for this coroutine attempt.

https://www.droidcon.com/2022/09/22/design-of-kotlin-coroutines/

This then goes one layer deper, it covers a lot of those layers you see in kotlin's implementation and explains what they're there for.

https://github.com/JetBrains/kotlin/blob/master/compiler/backend/src/org/jetbrains/kotlin/codegen/coroutines/coroutines-codegen.md

Finally, the mega document. Goes in depth into the jvm codegen of kotlin's coroutines.