Cannot import deltalake with python by maturin build

[RobinLin666]

trafficstars

Environment

Delta-rs version:

master branch

Binding: python 0.13

Environment:

• Cloud provider:
• OS:
• Other:

---

Bug

What happened:

cannot import deltalake with python

(delta) ➜  python git:(main) ✗ python                                                                                                    [19:11:38]
Python 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import deltalake
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/work/delta-rs/python/deltalake/__init__.py", line 1, in <module>
    from ._internal import __version__ as __version__
ModuleNotFoundError: No module named 'deltalake._internal'
>>>

What you expected to happen:

How to reproduce it:

 7930  git clone https://github.com/delta-io/delta-rs.git
 7931  cd delta-rs/python
 7932  curl https://sh.rustup.rs -sSf | sh -s
 7933  source "$HOME/.cargo/env"
 7934  pip install maturin==0.14
 7935  maturin build --release --out wheels
 7936  pip install wheels/deltalake-0.13.0-cp37-abi3-manylinux_2_31_x86_64.whl
 7937  python

OS: Ubuntu18.04 with WSL, I've tried mac, all the other linux, and they all report this mistake.

More details:

[wjones127]

I believe what's happening if that you are in the delta-rs source code directory and Python is confusing the deltalake directory with the installed module. You can see here the Python file path is in the source directory, not your installed modules directory:

If you are building for local development, use make develop or maturin develop.

If you are building wheels, it should work if you run from a different directory.

[RobinLin666]

Hahaha, you are right! Thank you! :)

[RobinLin666]

I don't know why, but I can't do it again recently. Because I cut to the main branch. I no longer execute under the python folder, so I rule out the above reasons.

[roeap]

What maturin version are you using? Recently updated the requirement to ^1.

[r3stl355]

I am unable to re-create this but I'm on Mac. Tried with the same Python and Pip versions, even tried with older maturin (0.15.3)

[roeap]

even tried with older maturin (0.15.3)

I would expect this to show up for maturin < 0.15, b/c the way the module name is specified changed there (from cargo.toml to pyprojet.toml. Had a similar message when updating so that's where the suspicion comes from ... 0.15 seems to be more or less in line with 1.0, at least there is no migration guide :)

[r3stl355]

yes, I managed to get the same error with maturin==0.13.6. It also give a warning during the build ⚠️ Warning: Couldn't find the symbol PyInit_deltalakein the native library. Python will fail to import this module. If you're using pyo3, check that#[pymodule]usesdeltalake as module name