Goroutines with wait-free communication

[nin-jin]

Description

std.cocurrency has fibers, threads, logical threads, sandboxes. But:

• It uses mutexes (and locks).
• fibers have large stack size that can't be resized.

std.parallelism has tasks, thread pool. But:

• tasks can't be suspended.

vide-core reimplements it: suspendable tasks that runs on thread pool. But:

• nonstandard lib.
• slow.
• It uses mutexes (and locks).
• tasks have large stack size that can't be resized.

golang has native support of stackfull coroutines with autosizable stacks that run on thread poll and communication through channels. But:

• It's another, dumb language.
• It doesn't support statically typed thread safety.
• It uses mutexes (and locks).

I wrote go.d to make go-like API with gorouties that run on the thread pool and communicates with wait-free channels. It uses vibe-core so inherits all it's cons. I tried to reimplement it on std.parallelism, on std.concurrency and on both, but suffer defeat. It's hard for me. And I need help.

I think we should have std api's to do it:

1. The simple way to run suspendable tasks on threadpool.
2. Tasks should have a small stack size with automatic growth like in go lang.
3. Locking should be reduced as more as possible.
4. Tasks should use wait-free gc-free channels to send data (and other tasks too).
5. Only immutable, shared and movable data should be transferrable through the channel.
6. Implementation should be as easy as possible.

What are rough milestones of this project?

0. Currently, there is a very simple code with remarks, tests and some diagrams.
1. Remove vibe-core dependency from go.d and implement all required functionality based on std.concurrency.
2. Update std to better support move semantic (ie we can't use sum with non-copiable ranges).
3. Remove duplicated abstractions from std (tasks, logical thread, fibers) - all abstractions should be orthogonal and composable.
4. Reimplement vibe-core API based on new std api.

How does this project help the D community?

It gives a standard simple way to make safe and fast multithreading apps.

Recommended skills

• Multithreading.
• Stack and heap at low-level.
• Go lang.
• Ownership and borrowing.

What can students expect to get out of doing this project?

• Respect of community for solving important D problems at the modern world.

Point of Contact

@nin-jin - tell me if I can help you to understand the ideas and join the project.

References

• Discussion about go.d.
• Article about transition from go lang.