pyjuliapkg icon indicating copy to clipboard operation
pyjuliapkg copied to clipboard

Empty compat intersection after Julia v1.12 upgrade (OpenSSL_jll)

Open sstroemer opened this issue 1 month ago • 2 comments

We've tried upgrading one of our tools to Julia v1.12.1 resulting in the following error when using juliacall:

Exception: 'julia' compat entries have empty intersection:
- '^1' at project_folder/.venv/lib/python3.13/site-packages/juliapkg/juliapkg.json
- '1.0.0 - 1.11' at project_folder/.venv/lib/python3.13/site-packages/juliacall/juliapkg.json (OpenSSL_jll)
- '~1.9, ^1.10.3' at project_folder/.venv/lib/python3.13/site-packages/juliacall/juliapkg.json
- '=1.12.1' at project_folder/juliapkg.json

This seems to trace back to the "dynamic" version specifier used in juliacall for OpenSSL_jll, which is - if I understand it correctly - resolved in juliapkg:

"version": "<=python"

Changing this, e.g. for Python 3.13, to 3.5.1 "fixes" this.

Can we change/rework this "version lookup" (not sure how it works internally) to improve the usability for Julia versions that clash with the 1.0.0 - 1.11 constraint?

sstroemer avatar Oct 28 '25 13:10 sstroemer

This behaviour is intentional. It arises because the version of Python you are using is shipped with OpenSSL 3.4 or older, whereas Julia 1.12 is shipped with OpenSSL 3.5. Using these together is likely to result in errors anywhere that you use OpenSSL (e.g. interact with the internet) so JuliaPkg ensures that the Julia version chosen results in compatible versions of this library.

The solution is to use a version of Python built with a newer version of OpenSSL. I believe the conda-forge versions of Python are built with OpenSSL 3.5 and so are compatible with Julia 1.12.

cjdoris avatar Oct 29 '25 20:10 cjdoris

That makes sense for me, from a "why it is the way it is" perspective, but it prevents most of our users from upgrading to 1.12, since they (as many others in the meantime) have switched to using uv as their "just make it work" solution. The problem arises, e.g., with the automatically installed CPython 3.13.7 version, so I'd like to find some kind of workaround in our package instead of trying to explain to users why their "up to date" Python does need a "fiddly" change.

So, maybe you have an idea on a possible workaround that we could do (that's a bit less hacky then overwriting the juliapkg.json file inside the virtual environment)? I've included some steps to reproduce below.


Creating a blank new project (e.g., uv init) and adding juliacall (via uv add juliacall) leads to these (or similar; depending on the exact 3.13 Python version) versions after running uv sync:

Using CPython 3.13.7
Creating virtual environment at: .venv
Resolved 7 packages in 2ms
Installed 6 packages in 4ms
 + filelock==3.20.0
 + juliacall==0.9.28
 + juliapkg==0.1.22
 + semver==3.0.4
 + tomli==2.3.0
 + tomlkit==0.13.3

Then just trying to execute import juliacall will fail with:

[juliapkg] Found dependencies: /home/user/dev/juliapkg.json
[juliapkg] Found dependencies: /home/user/dev/.venv/lib/python3.13/site-packages/juliapkg/juliapkg.json
[juliapkg] Found dependencies: /home/user/dev/.venv/lib/python3.13/site-packages/juliacall/juliapkg.json
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    import juliacall
  File "/home/user/dev/.venv/lib/python3.13/site-packages/juliacall/__init__.py", line 311, in <module>
    init()
    ~~~~^^
  File "/home/user/dev/.venv/lib/python3.13/site-packages/juliacall/__init__.py", line 178, in init
    CONFIG['exepath'] = juliapkg.executable()
                        ~~~~~~~~~~~~~~~~~~~^^
  File "/home/user/dev/.venv/lib/python3.13/site-packages/juliapkg/deps.py", line 611, in executable
    resolve()
    ~~~~~~~^^
  File "/home/user/dev/.venv/lib/python3.13/site-packages/juliapkg/deps.py", line 476, in resolve
    compat, pkgs = find_requirements()
                   ~~~~~~~~~~~~~~~~~^^
  File "/home/user/dev/.venv/lib/python3.13/site-packages/juliapkg/deps.py", line 420, in find_requirements
    raise Exception(
    ...<5 lines>...
    )
Exception: 'julia' compat entries have empty intersection:
- '^1.12' at /home/user/dev/juliapkg.json
- '^1' at /home/user/dev/.venv/lib/python3.13/site-packages/juliapkg/juliapkg.json
- '1.0.0 - 1.11' at /home/user/dev/.venv/lib/python3.13/site-packages/juliacall/juliapkg.json (OpenSSL_jll)
- '~1.9, ^1.10.3' at /home/user/dev/.venv/lib/python3.13/site-packages/juliacall/juliapkg.json

sstroemer avatar Nov 03 '25 18:11 sstroemer

I think the solution is to just wait a bit, or relax your constraints on Julia - do you really need Julia 1.12?

It looks like the Python builds that are used by default in uv now use OpenSSL 3.5, the same as Julia 1.12. I don't know what versions of Python they have built this way but it seems at least the latest Python 3.14 should work? I've not verified.

https://github.com/astral-sh/python-build-standalone/issues/775

cjdoris avatar Nov 14 '25 16:11 cjdoris