Is Pyjulia thread safe now?

[p-oorja]

We are working on SaaS based platform implemented using Flask and Celery. In the backend, we have some part of the implementation written in julia. We integrated julia using a python library called julia==0.6.0 In the flask we are able to run the julia code. However we are not able to run the same code inside the celery task. The error that we get is

celery_worker_1 | signal (4): Illegal instruction celery_worker_1 | in expression starting at none:2 celery_worker_1 | unknown function (ip: 0xffff727d6cec) celery_worker_1 | _jl_invoke at /cache/build/default-awsarm-5/julialang/julia-release-1-dot-8/src/gf.c:2365 [inlined] celery_worker_1 | ijl_apply_generic at /cache/build/default-awsarm-5/julialang/julia-release-1-dot-8/src/gf.c:2547 celery_worker_1 | Allocations: 3462340 (Pool: 3459082; Big: 3258); GC: 4 celery_worker_1 | [2022-12-18

You can replicate the error using the following repository: https://github.com/p-oorja/p-oorja

We are using julia v1.8.3 (even we have tried v1.9.0-alpha1 but we are getting the same error).

Is there any workaround to use pyjulia in threaded environment?

[mkitti]

I'm a bit confused about what you are doing and how celery works.

There is certainly a way interact with Julia in a thread safe way, but I'm not sure why you would want to do it via pyjulia.

Could you bring your question to Julia Discourse? https://discourse.julialang.org/

Also see this documentation on embedding Julia: https://docs.julialang.org/en/v1/manual/embedding/

https://docs.julialang.org/en/v1/devdocs/init/