tractor
tractor copied to clipboard
`ray` interop?
Per https://github.com/python-trio/trio/issues/2135#issuecomment-934109228 and #359 it'd be kinda interesting to try out a couple things:
- the possibility of embedding our runtime in a
@ray.remotethread- further, adding support to potentially swap in the
ray(c++) spawning layer instead of using OS process APIs B)
- further, adding support to potentially swap in the
- attempting
infect_asyncio=Truestyle inter-loop streaming to allow driving a remoteasynciotask for whatever benefits it provides (if any): https://docs.ray.io/en/latest/ray-core/actors/async_api.html?highlight=async%20for#asyncio-for-remote-tasks
Relvant ray related "actor" resources:
- blog post on retry, persistence and scaling: https://mpolinowski.github.io/docs/IoT-and-Machine-Learning/AIOps/2023-01-26-python-ray-remote-actors/2023-01-26/
- (pulled from
https://github.com/goodboy/tractor/issues/85#issuecomment-692804598)
and that goes through comparisons with
multiprocessing, though they're using fancy c++ threads + shm :joy: to "out-do"mp: https://towardsdatascience.com/10x-faster-parallel-python-without-python-multiprocessing-e5017c93cce1 - a "crash course" blog post: https://ray-project.github.io/q4-2021-docs-hackathon/0.4/ray-examples/ray-crash-course/02-Ray-Actors/
From their actual docs:
- actors api: https://docs.ray.io/en/latest/ray-core/actors.html
- particularly so called "async actors": https://docs.ray.io/en/latest/ray-core/actors/async_api.html#defining-an-async-actor
- tasks-in-workers api: https://docs.ray.io/en/latest/ray-core/tasks.html