typedb-driver icon indicating copy to clipboard operation
typedb-driver copied to clipboard

Investigate Python driver dependency on libpython

Open flyingsilverfin opened this issue 1 year ago • 0 comments

Problem to Solve

When using Python driver installations on Ubuntu/Debian, it works fine if you're using the system's built-in Python version, but starts to fall over if you apt install a newer python version (like python3.11 when you have python3.10 installed in your OS).

This may be a relatively common pattern as our driver is packaged for each python version.

The expected usage pattern combined with virtualenv. This example is taken from a set up on Ubuntu 20.04, which has Pyhton 3.8 installed by default but we want to use Python 3.11:

  1. install python3.11: sudo apt install python3.11
  2. install virtualenv via python python3.11 -m pip install virtualenv
  3. set up a virtualenv with that python: python3.11 -m virtualenv -p $(which python3.11) python311-venv
  4. activate the venv: source ./python311-venv/bin/activate
  5. Run python3 and try to use the driver

This will end up with an error like

>>> from typedb.driver import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/joshua/python-testing/testing-py311/lib/python3.11/site-packages/typedb/driver.py", line 27, in <module>
    from typedb.api.concept.concept import *  # noqa # pylint: disable=unused-import
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/joshua/python-testing/testing-py311/lib/python3.11/site-packages/typedb/api/concept/concept.py", line 28, in <module>
    from typedb.common.exception import TypeDBDriverException, INVALID_CONCEPT_CASTING
  File "/home/joshua/python-testing/testing-py311/lib/python3.11/site-packages/typedb/common/exception.py", line 26, in <module>
    from typedb.native_driver_wrapper import TypeDBDriverExceptionNative
  File "/home/joshua/python-testing/testing-py311/lib/python3.11/site-packages/typedb/native_driver_wrapper.py", line 10, in <module>
    from . import native_driver_python
ImportError: libpython3.11.so.1.0: cannot open shared object file: No such file or directory

To get around this one also needs to install the Python 3.11 shared library via apt: apt install python3.11-dev.

Proposed Solution

While this is okay, it's not a great UX. We should investigate whether we can remove the link-time dependency on the Python library itself.

flyingsilverfin avatar Nov 13 '23 10:11 flyingsilverfin