`rust` tasks

[goodboy]

Extremely interested in the idea of supporting direct spawn of async rust tasks (maybe in a diff thread) in our actor tree and even further eventually having a rust-native nursery API both for parallel threads and possibly (with less initial priotiy) async IO tasks.

Given python and rust seem to be best friends these days i'm thinking this would be a mega killer feature: https://github.com/PyO3/pyo3#examples

Ideally we get to a point where you easily convert some async python function into a rust equivalent without a ton of effort and allow more or less running rust based parallel tasks in-actor-tree B)

---

Research launch pad:

• there already exists py03 "native" support for asyncio which could be a place to start: https://github.com/awestlake87/pyo3-asyncio:

  • we could try a prototype where we run the subactor in infected_asyncio mode and see how this binding layer works,
  • details of asyncio event loop integration https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/
  • eventually we'll likely want our own direct integration with trio's runtime, which IIRC, njsmith said wouldn't be that bad :surfer:

• rustpython

  • there's an experimental jit in the works

• scoped threads APIs:

  • https://doc.rust-lang.org/stable/std/thread/fn.scope.html
  • https://docs.rs/crossbeam/latest/crossbeam/thread/index.html
  • https://github.com/rayon-rs/rayon
    • https://docs.rs/rayon/latest/rayon/fn.scope.html
    • blog posts from the creator of the rayon from above
      • https://smallcultfollowing.com/babysteps/blog/2023/05/09/giving-lending-and-async-closures/
      • https://smallcultfollowing.com/babysteps/blog/2023/03/03/trait-transformers-send-bounds-part-3/

• nursery APIS: https://github.com/nikomatsakis/moro#are-there-other-async-nursery-projects-available-and-how-does-moro-compare

• shared mem libs:

  • https://doc.rust-lang.org/book/ch16-03-shared-state.html?search=shared%20memory
  • https://crates.io/crates/memfd
  • https://lib.rs/crates/shared_hashmap
  • https://lib.rs/crates/shmem-ipc
  • https://users.rust-lang.org/t/rust-and-shared-memory/50783
  • https://stackoverflow.com/questions/76799889/allocating-and-managing-shared-memory-and-data-structures-in-rust
  • https://betterprogramming.pub/easy-multi-threaded-shared-memory-in-rust-57344e9e8b97
  • https://docs.rs/shared_memory/latest/shared_memory/
  • https://github.com/elast0ny/shared_memory/blob/master/examples/basic.rs
    • https://github.com/elast0ny/raw_sync-rs
  • https://github.com/Supportfactory/ringbuf/blob/master/ringbuf/src/lib.rs