nimpy icon indicating copy to clipboard operation
nimpy copied to clipboard

How to use it with multithreading?

Open cvanelteren opened this issue 1 year ago • 1 comments

I am playing around with Nim and python where I need to access python objects from different threads. In cython we get a handy with gil operator. Is there a way to access python objects in a similar way under Nimpy? Could you provide a minimal example?

Cheers! C

cvanelteren avatar Jul 15 '24 17:07 cvanelteren

In particular I have this example I am running with:

code

import nimpy{.all.}, dynlib
import nimpy/py_lib {.all.}
import malebolgia

type PyGILState_STATE = enum PyGILState_LOCKED, PyGILState_UNLOCKED

{.pragma: pyfunc, cdecl, gcsafe.}
# initPyLibIfNeeded()
let
    m = pythonLibHandleForThisProcess()
    nx = pyImport("networkx")
    PyGILState_Ensure = cast[proc(): PyGILState_STATE {.pyfunc.}](m.symAddr("PyGILState_Ensure"))
    PyGILState_Release = cast[proc(s: PyGILState_STATE) {.pyfunc.}](m.symAddr("PyGILState_Release"))

template withGIL*(code) =
    let state = Py_GILState_Ensure()
    code
    Py_GILState_Release(state)

withGil:
    echo "in gil"

proc createG(idx: int) =
  echo idx
  withGil:
    var g = nx.path_graph(3)

var master = createMaster()
master.awaitAll:
  for idx in 0..1000:
    master.spawn createG(idx)
echo "done"
unloadLib(m)

cvanelteren avatar Jul 15 '24 17:07 cvanelteren

Figured it out, see for explanation here

cvanelteren avatar Sep 06 '24 08:09 cvanelteren

Just to clarify for future reader since the link is incorrect -> This is the correct link https://thefriendlyghost.nl/unleashedgil/

Clonkk avatar Dec 06 '24 00:12 Clonkk

My bad updated my website in the meantime ;-) Thanks!

cvanelteren avatar Dec 06 '24 00:12 cvanelteren

It's a great article that would be deserving of being added to this repo as a test / example or documentation (IMO)

Clonkk avatar Dec 06 '24 00:12 Clonkk

Many thanks @Clonkk! Happy to provide a concrete test if @yglukhov wants to

cvanelteren avatar Dec 06 '24 01:12 cvanelteren

@cvanelteren absolutely! Please do!

yglukhov avatar Dec 06 '24 01:12 yglukhov

See #307

cvanelteren avatar Dec 06 '24 22:12 cvanelteren