core icon indicating copy to clipboard operation
core copied to clipboard

Improve single thread path for GIL (Python Loader) when executing any Python C API call

Open viferga opened this issue 3 years ago • 0 comments

🚀 Feature

Related to this issue: https://github.com/metacall/core/issues/266 ; it's is possible that single threaded execution may not need to save and restore thread context for each call. This is similar to when NodeJS executes the call in the same V8 thread, it does not need to be enqueued, it is safe to execute the call directly. For this, we can check if the GIL is acquired in the current thread and avoid the save and restore for each one.

I have detailed a bit of info in the Python tests that the previous issue has solved: https://github.com/metacall/core/blob/dae8f567fa7d1b189d292aaac32f0213e7e27ecb/source/ports/py_port/test/test_simple.py#L68

We should review this, it is not important but it may affect the performance. For benchmarking it, we can use ctest -VV -R metacall-py-call-bench

This has not high priority but it's a nice to have.

In order to understand what I am talking about, you can check this diff: https://github.com/metacall/core/commit/aa155a5c0f700b97d61b3ec8a26f6531ccc6b907#diff-838c25f4099cb483f5cbc18c530c64d637d27e86b761365fac17b0c0097d02b1

This shows what I have done in order to solve the bug of deadlock. Basically, those operations are only needed when we are executing the call from a different thread that the Python interpreter was created. So for single thread execution we can avoid them.

viferga avatar Apr 27 '22 00:04 viferga