exo-explore
docs: Contribute more documentation
Would love to help with more documentation for the project. I see the docs/ folder does not contain that much.
Are you guys open for contributions there?
What areas do you think should be covered first?
Absolutely - the docs repo was an artefact from the old repo's readme but we should start something new.
Just adding some thoughts here, exo's networking philosophy is fairly specific. We're writing some blog posts about it, but to summarise, exo's main components are:
- master: orders events for strong eventual consistency
- api: prepares state for client applications
- worker: schedules distributed work
- runner: ML specific code
- election: entrypoint into the clustering and determines a unique master
and as a follow up, our event sourcing model:
- events record a side effect (chunk generated, runner status)
- state is the summation of all events
- commands combine with state to trigger the creation of side effects
(task are a kind of command, they just go worker -> runner so don't get networked.) thinking aloud, we're definitely missing some kind of RPC abstraction to let us combine a command and the events it produces into a python async generator or similar.