libsql-experimental-python Bug Report: libsql-experimental build fails on Windows with Python 3.12

Duplicate: https://github.com/tursodatabase/libsql-experimental-python/issues/63. But more detailed on the error.

Bug Report: libsql-experimental build fails on Windows with Python 3.12

Description:

When attempting to install a package that depends on libsql-experimental on Windows with Python 3.12, the build process fails due to an error in the libsql-ffi build script.

The build process fails with the following error:

uv add libsql-experimental
Resolved 112 packages in 87ms
error: Failed to prepare distributions
  Caused by: Failed to download and build `libsql-experimental==0.0.41`
  Caused by: Build backend failed to build wheel through `build_wheel` (exit code: 1)

[stdout]
Running `maturin pep517 build-wheel -i C:\Users\PC\AppData\Local\uv\cache\builds-v0\.tmpmqnS34\Scripts\python.exe --compatibility off`

[stderr]
📦 Including license file "C:\Users\PC\AppData\Local\uv\cache\sdists-v5\pypi\libsql-experimental\0.0.41\5XwiJnUIC1DRVfvR9ZjQq\libsql_experimental-0.0.41.tar.gz\LICENSE.md"
🔗 Found pyo3 bindings
🐍 Found CPython 3.12 at C:\Users\PC\AppData\Local\uv\cache\builds-v0\.tmpmqnS34\Scripts\python.exe
📡 Using build options features from pyproject.toml
   Compiling ring v0.17.8
   Compiling pyo3-build-config v0.19.2
   Compiling libsql-ffi v0.3.0 (https://github.com/tursodatabase/libsql/?rev=21f405b087b210734367fb1343ed436249c8dc10#21f405b0)
   Compiling libsql-sqlite3-parser v0.12.0 (https://github.com/tursodatabase/libsql/?rev=21f405b087b210734367fb1343ed436249c8dc10#21f405b0)
error: failed to run custom build command for `libsql-ffi v0.3.0 (https://github.com/tursodatabase/libsql/?rev=21f405b087b210734367fb1343ed436249c8dc10#21f405b0)`

Caused by:
  process didn't exit successfully: `C:\Users\PC\AppData\Local\uv\cache\sdists-v5\pypi\libsql-experimental\0.0.41\5XwiJnUIC1DRVfvR9ZjQq\libsql_experimental-0.0.41.tar.gz\target\release\build\libsql-ffi-53910dba132d728a\build-script-build` (exit code: 101)
  --- stdout
  cargo:rerun-if-changed=bundled/src/sqlite3.c
  cargo:rerun-if-changed=C:\Users\PC\AppData\Local\uv\cache\sdists-v5\pypi\libsql-experimental\0.0.41\5XwiJnUIC1DRVfvR9ZjQq\libsql_experimental-0.0.41.tar.gz\target\release\build\libsql-ffi-e718525dc38faed8\out/sqlite3mc/libsqlite3mc_static.a

  --- stderr
  thread 'main' panicked at C:\Users\PC\scoop\persist\rustup\.cargo\git\checkouts\libsql-311658d335deb3b1\21f405b\libsql-ffi\build.rs:48:10:
  called `Result::unwrap()` on an `Err` value: Error { kind: NotFound, message: "program not found" }
  stack backtrace:
     0:     0x7ff77a0e3f41 - std::backtrace_rs::backtrace::dbghelp64::trace
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\..\..\backtrace\src\backtrace\dbghelp64.rs:91
     1:     0x7ff77a0e3f41 - std::backtrace_rs::backtrace::trace_unsynchronized
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\..\..\backtrace\src\backtrace\mod.rs:66
     2:     0x7ff77a0e3f41 - std::sys::backtrace::_print_fmt
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\sys\backtrace.rs:66
     3:     0x7ff77a0e3f41 - std::sys::backtrace::impl$0::print::impl$0::fmt
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\sys\backtrace.rs:39
     4:     0x7ff77a105b39 - core::fmt::rt::Argument::fmt
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/core\src\fmt\rt.rs:177
     5:     0x7ff77a105b39 - core::fmt::write
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/core\src\fmt\mod.rs:1178
     6:     0x7ff77a0df4b7 - std::io::Write::write_fmt<std::sys::pal::windows::stdio::Stderr>
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\io\mod.rs:1823
     7:     0x7ff77a0e6309 - std::panicking::default_hook::closure$1
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\panicking.rs:266
     8:     0x7ff77a0e5e8c - std::panicking::default_hook
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\panicking.rs:293
     9:     0x7ff77a0e6c12 - std::panicking::rust_panic_with_hook
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\panicking.rs:797
    10:     0x7ff77a0e6a56 - std::panicking::begin_panic_handler::closure$0
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\panicking.rs:671
    11:     0x7ff77a0e49ff - std::sys::backtrace::__rust_end_short_backtrace<std::panicking::begin_panic_handler::closure_env$0,never$>
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\sys\backtrace.rs:170
    12:     0x7ff77a0e6666 - std::panicking::begin_panic_handler
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\panicking.rs:662
    13:     0x7ff77a10eae4 - core::panicking::panic_fmt
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/core\src\panicking.rs:74
    14:     0x7ff77a10ef00 - core::result::unwrap_failed
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/core\src\result.rs:1677
    15:     0x7ff779b7460e - enum2$<core::result::Result<std::process::Output,std::io::error::Error> >::unwrap
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library\core\src\result.rs:1102
    16:     0x7ff779b7460e - build_script_build::main
                                 at C:\Users\PC\scoop\persist\rustup\.cargo\git\checkouts\libsql-311658d335deb3b1\21f405b\libsql-ffi\build.rs:42
    17:     0x7ff779b7201b - core::ops::function::FnOnce::call_once<void (*)(),tuple$<> >
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library\core\src\ops\function.rs:250
    18:     0x7ff779b71a6e - core::hint::black_box
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library\core\src\hint.rs:388
    19:     0x7ff779b71a6e - std::sys::backtrace::__rust_begin_short_backtrace<void (*)(),tuple$<> >
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library\std\src\sys\backtrace.rs:154
    20:     0x7ff779b71921 - std::rt::lang_start::closure$0<tuple$<> >
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library\std\src\rt.rs:164
    21:     0x7ff77a0dad99 - std::rt::lang_start_internal::closure$2
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\rt.rs:143
    22:     0x7ff77a0dad99 - std::panicking::try::do_call
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\panicking.rs:554
    23:     0x7ff77a0dad99 - std::panicking::try
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\panicking.rs:518
    24:     0x7ff77a0dad99 - std::panic::catch_unwind
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\panic.rs:345
    25:     0x7ff77a0dad99 - std::rt::lang_start_internal
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\rt.rs:143
    26:     0x7ff779b718fa - std::rt::lang_start<tuple$<> >
                                 at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library\std\src\rt.rs:163
    27:     0x7ff779b78d59 - main
    28:     0x7ff77a10bf00 - invoke_main
                                 at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
    29:     0x7ff77a10bf00 - __scrt_common_main_seh
                                 at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
    30:     0x7ff875bd257d - BaseThreadInitThunk
    31:     0x7ff87736af08 - RtlUserThreadStart
warning: build failed, waiting for other jobs to finish...
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit code: 101": `"cargo" "rustc" "--features" "pyo3/extension-module" "--message-format" "json-render-diagnostics" "--manifest-path" "C:\\Users\\PC\\AppData\\Local\\uv\\cache\\sdists-v5\\pypi\\libsql-experimental\\0.0.41\\5XwiJnUIC1DRVfvR9ZjQq\\libsql_experimental-0.0.41.tar.gz\\Cargo.toml" "--release" "--lib"`
Error: command ['maturin', 'pep517', 'build-wheel', '-i', 'C:\\Users\\PC\\AppData\\Local\\uv\\cache\\builds-v0\\.tmpmqnS34\\Scripts\\python.exe', '--compatibility', 'off'] returned non-zero exit status 1

Steps to reproduce:

Set up a Python 3.12 environment on Windows 11.
Attempt to install a package that depends on libsql-experimental, for example, using pip install <package_name>.

Environment:

Operating System: Windows 11
Python: 3.12
Cargo version: 1.82.0 (8f40fc59f 2024-08-21)
CMake: version 3.30.5
Clang version 19.1.2

Possible cause:

The error message "program not found" suggests that the libsql-ffi build script is failing to find a required program on the system. This could be due to a missing dependency or an incorrect path configuration.

Labels:

bug
windows
python3.12
build

Oct 26 '24 04:10 minhquan23102000

Any help, really hope this will fix soon. Current the alternative is I use pure sqlite.

Oct 26 '24 04:10 minhquan23102000

I ran it with Dockerfile if that helps:

FROM python:3-slim-bookworm AS builder

WORKDIR /app

RUN apt -yqq update
RUN apt -yqq install curl build-essential gcc make cmake
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y

COPY requirements.txt .
RUN . ~/.cargo/env && pip install --user -r requirements.txt

###############################################################################
FROM python:3-slim-bookworm

WORKDIR /app

COPY --from=builder /root/.local /root/.local
ENV PATH=/root/.local/bin:$PATH

COPY . .

# Make entrypoint script executable
RUN chmod +x docker-entrypoint.sh

HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 CMD curl -f http://localhost:8000/health

ENTRYPOINT ["/app/docker-entrypoint.sh"]

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Dec 22 '24 18:12 andrius

I think this specific error was due to the use of "cp" to copy files in libsql-ffi's build.rs file (line 48) which isn't present natively in windows. It looks like that bit of code has been updated in the source repository to handle different OSes now https://github.com/tursodatabase/libsql/blob/d42e05a5f092f957b81c9b6f6cf9aa794652d9be/libsql-ffi/build.rs#L60

That said, I also ran into issues building another dependency ([email protected]) on windows and gave up at that point (not sure if that's an environmental problem on my part but we're talking about python building rust building C at this point, so I'm out :D). I noticed later that the readme for this repo does say it only supports Linux and MacOS though, so, there's that.

Mar 01 '25 02:03 emilkloeden

fork my branch and then run it in action, you will see in artifact

Mar 26 '25 18:03 swoiow

@andrius I've found with python:3.11.11-slim-bullseye that build-essential and cmake seem to be sufficient when installing via uv

i.e.

$ docker run -it python:3.11.11-slim-bullseye bash

> apt -y install && apt -y upgrade && apt -y curl cmake build-essential
> curl -LsSf https://astral.sh/uv/install.sh | sh # install uv
> . $HOME/.local/bin/env # make uv available in path
> uv init test && cd test
> uv add libsql # installs v0.1.2

Jul 03 '25 12:07 larrybotha