diffeqpy
diffeqpy copied to clipboard
SciML
Error when importing diffeqpy parallely in large number of sub-processes.
I have a program that spawns a large number of sub-processes and imports diffeqpy within each sub-process using:
from diffeqpy import ode
It works fine when the number of sub-processes is less than 75. But above that I am getting the following error:
"exception: access violation writing 0x000001EC9127C000"
I am using the following: diffeqpy: 1.2.0 pyjulia: 0.5.7 Julia 1.5.3, 1.6, 1.7
I have a feeling this could be a pyjulia problem. But I just wanted to check here to see if anyone else has encountered it.
Sounds like a pyjulia problem. I'm not sure we can solve it at this repo.
@ChrisRackauckas I tried an experiment where I only imported the Main function using pyjulia with many subprocesses. In this case, I was able to start more than 75 sub-processes without encountering the error. I checked the memory footprints of the sub-processes. It seems to importing diffeqpy almost double the memory footprint of the Main function. Could the higher memory footprint when using diffeqpy be the cause of the problem?
Is it dying because you run out of memory? That would be a good way to diagnose that 😅
@ChrisRackauckas No, the workstation I am using has 128 GB of memory. And only about 50% memory utilization is there at the time of the error.
Maybe you are using fork? See: https://github.com/JuliaPy/pyjulia/issues/448#issuecomment-994965548
Has this issue been resolved? I am trying to use the diffeqpy package in a general fitting software (PyBNF) but without the dask compatibility I am getting nowhere. If I try to use julia prior to submitting the jobs I get a error trying to pickle pycall objects. If I import julia in the function, it is extremely slow and prone to crashing.
@JacobNeumann88 No my issue is not resolved. So in my application, I am limited to using less than 50 instances of diffeqpy. I have not used it with Dask.
If I import julia in the function, it is extremely slow and prone to crashing.
I am importing diffeqpy within each sub-process. Depending upon the number of sub-processes it takes anywhere from 10 seconds to 20 seconds. And if I have more than 50 sub-processes my program crashes (not immediately)