Moar spawning backends (`rsyscall`, `nogil` threads)

[goodboy]

2 major projects on the radar:

rsyscall:

The project I'm personally most excited about. https://github.com/catern/rsyscall

It's a a fully distributed syscall layer that can be used to remotely spawn pythons over SSH with built-in tunneled TCP connections 😎

A lot of these notes might get moved into a separate issue as work starts on this.

NOTES:

• the high level process spawning api: Thread.clone() and wrapping Command api
• persistent threads via rsyscall.tasks.persistent
• the boostrapped Thread over ssh api: rsyscall.tasks.ssh
• a PR that merged proc-title control like py-setproctitle
• the "wish" system as a possible basis for a properly built async-aware repl-debugger
• test exemplifying nested Threads via a trio nursery

Subsystems of interest:

• dneio: Concurrency based on shift/reset and object-oriented effect handlers
  • main pkg-mod with big doc-string: https://github.com/catern/rsyscall/blob/master/python/dneio/init.py
  • core impl: https://github.com/catern/rsyscall/blob/master/python/dneio/core.py

Updates:

• checkout the new mem write pipelining

Must read's from the author:

• http://catern.com/supervisors.html
• http://catern.com/integration.html
• http://catern.com/microkernels.html
• http://catern.com/computers.html

---

nogil cpython:

A nogil fork which we should be compat with out of the box minus whatever fork-like issues trio still has:

• design doc on gdocs
• this should make localhost spawning super fast since, threads, but only as long as we Thread.start() + trio.to_thread.run() style?
• trio still can't do forking without mucking the run loop state
• apparently asyncio got faster?
• also, it's legit going to land in cpython in a major release at some point :sunglasses: https://discuss.python.org/t/a-steering-council-notice-about-pep-703-making-the-global-interpreter-lock-optional-in-cpython/30474/20
• required reading on threads vs. processes for linux:
  • https://www.baeldung.com/cs/process-vs-thread
    • https://www.baeldung.com/linux/process-vs-thread
  • https://man7.org/linux/man-pages/man2/clone.2.html
  • http://catern.com/process.html#org2dbcddb
    • http://catern.com/supervisors.html