PythonCall.jl icon indicating copy to clipboard operation
PythonCall.jl copied to clipboard
verified publisher icon JuliaPy

rare ijl_bounds_error_int with Julia 1.11 & python 3.13

Open dpinol opened this issue 7 months ago • 4 comments

Affects: JuliaCall

Describe the bug Very rarely, my python application crashes while juliacall is invoking a julia function.

This is the error I get:

error in running finalizer: BoundsError(a=MemoryRef(offset=0, ptr_or_offset=0x5a44f85dd300, mem=Memory{Int64}(1819, 0x5a44f85dd300)[....]

ijl_bounds_error_int at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/rtutils.c:207
memoryref at ./boot.jl:524 [inlined]
unsafe_copyto! at ./genericmemory.jl:115 [inlined]
#133 at ./array.jl:1133
_growend! at ./array.jl:1116 [inlined]
push! at ./array.jl:1261 [inlined]
_pyjl_dealloc at /home/avatar/.julia/packages/PythonCall/hXycg/src/JlWrap/C.jl:35
jfptr__pyjl_dealloc_9930 at /home/avatar/.julia/compiled/v1.11/PythonCall/WdXsa_ttmfG.so (unknown line)
jlcapi__pyjl_dealloc_10065 at /home/avatar/.julia/compiled/v1.11/PythonCall/WdXsa_ttmfG.so (unknown line)
subtype_dealloc at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:2388
Py_DecRef at /home/avatar/.julia/packages/PythonCall/hXycg/src/C/pointers.jl:300 [inlined]
enqueue at /home/avatar/.julia/packages/PythonCall/hXycg/src/GC/GC.jl:99
py_finalizer at /home/avatar/.julia/packages/PythonCall/hXycg/src/Core/Py.jl:27
unknown function (ip: 0x7c5556d13607)
run_finalizer at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/gc.c:303
jl_gc_run_finalizers_in_list at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/gc.c:393
run_finalizers at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/gc.c:439
run_finalizers at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/gc.c:420 [inlined]
ijl_gc_collect at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/gc.c:3915
maybe_collect at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/gc.c:926 [inlined]
jl_gc_pool_alloc_inner at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/gc.c:1319
jl_gc_alloc_ at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/julia_internal.h:523 [inlined]
_new_genericmemory_ at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/genericmemory.c:56 [inlined]
jl_alloc_genericmemory at /cache/build/tester-amdci4-12/julialang/julia-release-1-dot-11/src/genericmemory.c:99
GenericMemory at ./boot.jl:516 [inlined]
array_new_memory at ./array.jl:1058 [inlined]
#133 at ./array.jl:1129
_growend! at ./array.jl:1116 [inlined]
push! at ./array.jl:1261 [inlined]
_pyjl_dealloc at /home/avatar/.julia/packages/PythonCall/hXycg/src/JlWrap/C.jl:35
jfptr__pyjl_dealloc_9930 at /home/avatar/.julia/compiled/v1.11/PythonCall/WdXsa_ttmfG.so (unknown line)
jlcapi__pyjl_dealloc_10065 at /home/avatar/.julia/compiled/v1.11/PythonCall/WdXsa_ttmfG.so (unknown line)
subtype_dealloc at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:2388
Py_DECREF at /tmp/python-build.20250826113541.3096764/Python-3.13.7/./Include/object.h:949 [inlined]
Py_XDECREF at /tmp/python-build.20250826113541.3096764/Python-3.13.7/./Include/object.h:1042 [inlined]
tupledealloc at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/tupleobject.c:208
Py_DECREF at /tmp/python-build.20250826113541.3096764/Python-3.13.7/./Include/object.h:949 [inlined]
Py_XDECREF at /tmp/python-build.20250826113541.3096764/Python-3.13.7/./Include/object.h:1042 [inlined]
list_dealloc at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/listobject.c:566
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:2885
_PyObject_VectorcallDictTstate at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:135
_PyObject_Call_Prepend at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:504
slot_tp_init at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:9816
type_call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:1997
_PyObject_Call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:361 [inlined]
_PyObject_Call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:333 [inlined]
PyObject_Call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:373
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:1355
_PyObject_VectorcallDictTstate at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:135
_PyObject_Call_Prepend at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:504
slot_tp_call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:9570
_PyObject_MakeTpCall at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:242
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:813
_PyObject_VectorcallDictTstate at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:135
_PyObject_Call_Prepend at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:504
slot_tp_init at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:9816
type_call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:1997
_PyObject_MakeTpCall at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:242
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:813
_PyObject_VectorcallDictTstate at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:146
_PyObject_Call_Prepend at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:504
slot_tp_init at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:9816
type_call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:1997
_PyObject_Call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:361 [inlined]
_PyObject_Call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:333 [inlined]
PyObject_Call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:373
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:1355
_PyObject_VectorcallDictTstate at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:135
_PyObject_Call_Prepend at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:504
slot_tp_init at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:9816
type_call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:1997
_PyObject_MakeTpCall at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:242
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:813
_PyObject_VectorcallDictTstate at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:135
_PyObject_Call_Prepend at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:504
slot_tp_init at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:9816
type_call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:1997
_PyObject_MakeTpCall at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:242
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:813
_PyEval_EvalFrame at /tmp/python-build.20250826113541.3096764/Python-3.13.7/./Include/internal/pycore_ceval.h:119 [inlined]
_PyEval_Vector at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/ceval.c:1816 [inlined]
PyEval_EvalCode at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/ceval.c:604
run_eval_code_obj at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pythonrun.c:1381 [inlined]
run_eval_code_obj at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pythonrun.c:1348
run_mod at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pythonrun.c:1466
pyrun_file at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pythonrun.c:1295 [inlined]
_PyRun_SimpleFileObject at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pythonrun.c:517
_PyRun_AnyFileObject at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pythonrun.c:77
pymain_run_file_obj at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:410 [inlined]
pymain_run_file at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:429 [inlined]
pymain_run_python at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:696 [inlined]
Py_RunMain at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:775
pymain_main at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:805 [inlined]
Py_BytesMain at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:829
unknown function (ip: 0x7c555cc2a1c9)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at python (unknown line)

...My python code...
  File "/home/avatar/.julia/packages/PythonCall/hXycg/src/JlWrap/any.jl", line 262, in __call__
    return self._jl_callmethod($(pyjl_methodnum(pyjlany_call)), args, kwargs)
juliacall.JuliaError: <exception str() failed>

[1695082] signal 11 (1): Segmentation fault
in expression starting at none:0
delta_new at /tmp/python-build.20250826113541.3096764/Python-3.13.7/./Modules/_datetimemodule.c:2745
type_call at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:1985
_PyObject_MakeTpCall at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:242
_PyEval_EvalFrameDefault at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/generated_cases.c.h:1502
_PyObject_VectorcallTstate at /tmp/python-build.20250826113541.3096764/Python-3.13.7/./Include/internal/pycore_call.h:168 [inlined]
PyObject_CallOneArg at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/call.c:395
call_unbound_noarg at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:2577 [inlined]
slot_tp_finalize at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Objects/typeobject.c:9863
finalize_garbage at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/gc.c:980 [inlined]
gc_collect_main at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/gc.c:1408
finalize_modules at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pylifecycle.c:1757
_Py_Finalize at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Python/pylifecycle.c:2134
Py_RunMain at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:777
pymain_main at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:805 [inlined]
Py_BytesMain at /tmp/python-build.20250826113541.3096764/Python-3.13.7/Modules/main.c:829
unknown function (ip: 0x7c555cc2a1c9)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at python (unknown line)
Allocations: 30653030 (Pool: 30651924; Big: 1106); GC: 26

Your system Please provide detailed information about your system:

  • The operating system: Ubuntu 25.04
  • Julia 1.11.6, python 3.13.5, PythonCall 0.9.27 , JuliaCall 0.9.27
  • If an issue with JuliaCall, the output of pip list or conda list and juliapkg.status().
juliacall               0.9.27
juliapkg                0.1.17

Julia 1.11.6 (=1.11.6) @ /home/dani/.julia/juliaup/julia-1.11.6+0.x64.linux.gnu/bin/julia
Packages:
  PythonCall: {'uuid': '6099a3de-0909-46bc-b1f4-468b9a2dfc0d'}
  SCEngine: {'uuid': 'bfeac117-3e8e-4b47-838f-ff82b809d5a4', 'dev': True, 'version': '=0.3.0', 'path': '/home/dani/source/repos/sc-engine'}
  SCCli: {'uuid': '274a597d-43d1-4cc6-8622-452e712fc982', 'dev': True, 'version': '=0.3.0', 'path': '/home/dani/source/repos/sc-cli'}
  SCRunner: {'uuid': 'ea12bd49-5513-406d-806f-9aeee9d9fc30', 'dev': True, 'version': '=0.3.0', 'path': '/home/dani/source/repos/sc-runner'}
  SCTestHelpers: {'uuid': 'd3adfe28-cc0d-41fe-9d43-3a9a094f5274', 'dev': True, 'version': '=0.3.0', 'path': '/home/dani/source/repos/sc-engine', 'subdir': 'sc-testhelpers'}

dpinol avatar Sep 16 '25 16:09 dpinol

Hmm. When we wrap Julia values into Python objects, the Julia object is stored in a global vector and referred to by index from the Python object. It seems that this vector has got corrupted somehow.

Do you have any multithreading? We don't use any thread-safe data structures for this, but then I thought the GIL should protect us.

cjdoris avatar Sep 16 '25 21:09 cjdoris

I have multithreading, but only on the julia side. So it shouldn't affect, right? I'll try to create MWE which eventually crashes, but not sure it will do it.

Could this be related to https://github.com/JuliaPy/PythonCall.jl/pull/637? Does it make sense that I try this branch?

dpinol avatar Sep 17 '25 12:09 dpinol

Honestly I don't know - just the nature of the error suggests the state is corrupted, which suggests some race condition. You could try that branch and see.

The state in question is these two vectors: https://github.com/JuliaPy/PythonCall.jl/blob/3633da04b7ac568cfcf10e459116aa171f9db830/src/JlWrap/C.jl#L17-L21

Similar to that #637, you could add a lock to protect any accesses to these two vectors. If the problem goes away then there is a race condition.

Then if so, you can then migrate to using trylock and print a debug message whenever the lock cannot be immediately acquired. This will find where the race condition would be hit if we didn't have locks.

cjdoris avatar Sep 17 '25 15:09 cjdoris

Still getting similar error with julia 1.12.2 and python 3.14.2

error in running finalizer: ConcurrencyViolationError(msg="Vector has invalid state. Don't modify internal fields incorrectly, or resize without correct locks")
#_growend!##0 at ./array.jl:1133
_growend! at ./array.jl:1131 [inlined]
_push! at ./array.jl:1289 [inlined]
push! at ./array.jl:1286 [inlined]
_pyjl_dealloc at /home/dani/.julia/packages/PythonCall/okl5Q/src/JlWrap/C.jl:35
jlcapi__pyjl_dealloc_10719 at /home/dani/.julia/compiled/v1.12/PythonCall/WdXsa_3UqGY.so (unknown line)

dpinol avatar Dec 11 '25 16:12 dpinol