PythonCall.jl icon indicating copy to clipboard operation
PythonCall.jl copied to clipboard

`import juliacall` sometimes segfaults

Open LilithHafner opened this issue 2 years ago • 3 comments

Affects: JuliaCall

Describe the bug I sometimes get segfaults when importing julicall in combination with other packages.

x@x ~ % python -c 'import juliacall; import torch; from torch.autograd import Function'
x@x ~ % python -c 'import torch; from torch.autograd import Function; import juliacall'
x@x ~ % python -c 'import torch; from torch.autograd import Function; import juliacall'
zsh: segmentation fault  python3 -c 
x@x ~ % python -c 'import torch; from torch.autograd import Function; import juliacall'
zsh: segmentation fault  python3 -c 
x@x ~ % python -c 'import torch; from torch.autograd import Function; import juliacall'
zsh: segmentation fault  python3 -c 
x@x ~ % python -c 'import torch; from torch.autograd import Function; import juliacall'
zsh: segmentation fault  python3 -c 

Your system

x@x ~ % python --version
Python 3.11.2
x@x ~ % julia -e 'using InteractiveUtils; versioninfo()'
Julia Version 1.9.3
Commit bed2cd540a1 (2023-08-24 14:43 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin22.4.0)
  CPU: 8 × Apple M2
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, apple-m1)
  Threads: 1 on 4 virtual cores
Environment:
  JULIA_EDITOR = code
  JULIA_PYTHONCALL_EXE = /Library/Frameworks/Python.framework/Versions/3.11/bin/python3
x@x ~ % python -m pip list                              
Package                   Version
------------------------- ---------
absl-py                   2.0.0
accelerate                0.23.0
aiohttp                   3.8.4
aiosignal                 1.3.1
astunparse                1.6.3
async-timeout             4.0.2
attrs                     22.2.0
cachetools                5.3.1
certifi                   2022.12.7
cffi                      1.15.1
charset-normalizer        3.1.0
click                     8.1.3
codespell                 2.2.6
comtypes                  1.1.14
contourpy                 1.0.7
cycler                    0.11.0
diffeqpy                  2.1.0
diffusers                 0.21.1
Faker                     19.10.0
filelock                  3.12.4
fire                      0.5.0
flatbuffers               23.5.26
fonttools                 4.39.3
frozenlist                1.3.3
fsspec                    2023.9.0
gast                      0.4.0
google-auth               2.23.0
google-auth-oauthlib      1.0.0
google-pasta              0.2.0
grpcio                    1.58.0
gTTS                      2.3.1
h5py                      3.9.0
huggingface-hub           0.17.1
idna                      3.4
importlib-metadata        6.8.0
jill                      0.11.5
Jinja2                    3.1.2
jsonschema                4.19.0
jsonschema-specifications 2023.7.1
juliacall                 0.9.14
juliapkg                  0.1.10
keras                     2.13.1
kiwisolver                1.4.4
libclang                  16.0.6
llvmlite                  0.41.0
Markdown                  3.4.4
MarkupSafe                2.1.3
matplotlib                3.7.1
mecab-python3             1.0.6
mpmath                    1.3.0
multidict                 6.0.4
nanoid                    2.0.0
networkx                  3.1
numba                     0.58.0
numpy                     1.24.2
oauthlib                  3.2.2
openai                    0.27.2
opencv-python             4.7.0.72
opt-einsum                3.3.0
packaging                 23.1
Pillow                    9.5.0
pip                       23.2.1
playsound                 1.3.0
protobuf                  4.24.3
psutil                    5.9.5
pyasn1                    0.5.0
pyasn1-modules            0.3.0
pycparser                 2.21
pyobjc-framework-Vision   9.0.1
pyparsing                 3.0.9
python-dateutil           2.8.2
python-gnupg              0.5.1
pyttsx3                   2.91
PyYAML                    6.0.1
referencing               0.30.2
regex                     2023.8.8
requests                  2.28.2
requests-futures          1.0.1
requests-oauthlib         1.3.1
rlvoice                   1.0.0
rpds-py                   0.10.2
rsa                       4.9
safetensors               0.3.3
semantic-version          2.10.0
setuptools                65.5.0
six                       1.16.0
sounddevice               0.4.6
srt                       3.5.3
sympy                     1.12
tensorboard               2.13.0
tensorboard-data-server   0.7.1
tensorflow                2.13.0
tensorflow-estimator      2.13.0
tensorflow-macos          2.13.0
termcolor                 2.3.0
tokenizers                0.13.3
torch                     2.0.1
tqdm                      4.65.0
transformers              4.33.1
typing_extensions         4.5.0
urllib3                   1.26.15
vosk                      0.3.44
websockets                10.4
Werkzeug                  2.3.7
wget                      3.2
wheel                     0.41.2
wrapt                     1.15.0
yarl                      1.8.2
zipp                      3.16.2
x@x ~ % julia                                           
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.3 (2023-08-24)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> juliapkg.status()
ERROR: UndefVarError: `juliapkg` not defined
Stacktrace:
 [1] top-level scope
   @ REPL[1]:1

julia> 
x@x ~ % python -c 'import juliacall; juliacall.juliapkg.status()'
┌ Warning: attempting to remove probably stale pidfile
│   path = "/Users/x/.julia/logs/manifest_usage.toml.pid"
└ @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.9.3+0.aarch64.apple.darwin14/share/julia/stdlib/v1.9/FileWatching/src/pidfile.jl:244
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: module 'juliacall' has no attribute 'juliapkg'

LilithHafner avatar Oct 19 '23 14:10 LilithHafner

Yeah it's a known issue. See also #215.

cjdoris avatar Oct 19 '23 23:10 cjdoris

That issue suggested printing a warning if torch is loaded before juliacall, which was never implemented.

cjdoris avatar Oct 19 '23 23:10 cjdoris