pyjulia icon indicating copy to clipboard operation
pyjulia copied to clipboard

How to do multi-threading within julia when using pyjulia

Open hqs811 opened this issue 4 years ago • 2 comments

In the documentation, it says

_PyJulia cannot be used in different threads since libjulia is not thread safe. However, you can use multiple threads within Julia. For example, start IPython by JULIA_NUM_THREADS=4 ipython and then run:

.... code ...._

Question: is there a way to enable a multi-thread Julia in a python script so that I can run it from the command line? Something like python3 call_julia.py

Where call_julia.py is a python script like below:

import julia from julia.api import Julia

may be something magic happen here so that imported Julia is multi-threaded?

from julia import Main jl = Julia(compiled_modules=False)

jl.eval('include("functions_defined_in_Julia.jl")')

def julia_model(parameters): Main.parameters = parameters x = jl.eval("julia_model(parameters)") return x

Any thoughts? I have read through the documentation but still got no idea how to do it, thanks!

hqs811 avatar Apr 08 '21 13:04 hqs811

Well, I do it this way:

import os
from multiprocessing import cpu_count
# read num of cpus and set the julia threas var
os.environ["JULIA_NUM_THREADS"] = str(cpu_count())

#import (py)julia
from julia import Main as jl

#do simething
jl.eval('some code')
jl.include("Main.jl")

smjaberl avatar Jun 01 '21 00:06 smjaberl

Whatever I try, nthreads is always 1. I have tried the answer above, I have also tried setting JULIA_NUM_THREADS in windows environment variables. A short example from the console below:

import os
os.environ["JULIA_NUM_THREADS"] = '4'
from julia import Main
Main.eval('Threads.nthreads()')
Out[5]: 1

Anyone any luck with this?

nleenders avatar Aug 10 '22 12:08 nleenders