Python and Multi-Threading

[eaplatanios]

Is there a way currently to use Python in multi-threaded contexts? I get segfaults whenever I try to do that and my guess is because the interpreter was started by a different thread. Would one possible solution be to make PythonInterface.init() public so that instances of the interpreter can be created from other threads?

cc @dan-zheng @rxwei

[avnerbarr]

Any news on this? Can we call from dispatch blocks?

[dabrahams]

The idea of starting separate interpreters is a reasonable one, though I'm pretty certain you can't share any python objects across these interpreters, and I don't know if that fits the use case of @eaplatanios

Otherwise, interacting from multiple non-python threads seems to involve acquiring the GIL.

Progress in this domain is not currently a short-term priority for the team but contributions are certainly welcome.

[rex-remind101]

I have an interest in adding this, however, I can't seem to find where the Python code actually lives.

[saeta]

@rex-remind101 the integration currently lives at https://github.com/pvieito/PythonKit

Context: this was copied from S4TF to make the Python interop available to all Swift users, without requiring a S4TF toolchain. As part of our work to upstream everything, we've now removed our (original) copy of the Python interop, and have moved to depending on the copy (stable link).

[rex-remind101]

Ah! Thanks @saeta that explains it!

[rex-remind101]

I've posted a topic for discussion https://groups.google.com/a/tensorflow.org/forum/#!topic/swift/MWFwGhHT36w