prefix-dev
Failing to build rpy2: cannot find -llzma: No such file or directory
Checks
-
[x] I have checked that this issue has not already been reported.
-
[x] I have confirmed this bug exists on the latest version of pixi, using
pixi --version.
Reproducible example
pyproject.toml:
[project]
name = "pymgcv"
authors = [{ name = "danielward27", email = "[email protected]" }]
description = "pymgcv: Generalized additive models using mgcv, in Python."
requires-python = ">= 3.11"
version = "0.0.0"
keywords = ["generalized additive models", "GAMs"]
license = { file = "LICENSE" }
dependencies = ["rpy2", "pytz", "pandas", "matplotlib", "numpy"]
[project.optional-dependencies]
dev = [
"pytest",
"beartype",
"pytest-beartype",
"ruff",
"hippogriffe",
"mkdocs",
"mkdocs-material",
"mkdocstrings[python]",
"pymdown-extensions",
"griffe-inherited-docstrings",
]
[build-system]
build-backend = "hatchling.build"
requires = ["hatchling"]
[tool.pytest.ini_options]
pythonpath = ["."]
beartype_packages = 'pymgcv'
# ------------------- pixi config --------------------#
[tool.pixi.dependencies]
r-base = "*"
r-mgcv = "*"
numpy = "*"
pandas = "*"
[tool.pixi.pypi-dependencies]
pymgcv = { path = ".", editable = true }
[tool.pixi.workspace]
channels = ["conda-forge"]
platforms = ["linux-64"]
[tool.pixi.tasks]
[tool.pixi.environments]
dev = ["dev"]
The .github.workflow is:
name: test
on:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
Test:
name: Tests
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: prefix-dev/[email protected]
with:
environments: dev
- run: pixi run -e dev pytest
The failing job is here: https://github.com/danielward27/pymgcv/actions/runs/14417117507/job/40435039339
Issue description
Locally, on a Ubuntu machine, my environment installs fine, e.g. the following works
git clone -b pixi_action https://github.com/danielward27/pymgcv.git
cd pymgcv
pixi shell --environment=dev
pytest
However, when installing on ubuntu-latest with a github action I get:
/home/runner/.pixi/bin/pixi install -e dev --locked --manifest-path pyproject.toml --color always
Error: × Failed to update PyPI packages for environment 'dev'
├─▶ Failed to prepare distributions
├─▶ Failed to build `rpy2==3.5.17`
├─▶ The build backend returned an error
╰─▶ Call to `setuptools.build_meta.build_wheel` failed (exit status: 1)
[stderr]
/home/runner/work/pymgcv/pymgcv/.pixi/envs/dev/compiler_compat/ld:
cannot find -llzma: No such file or directory
collect2: error: ld returned 1 exit status
Traceback (most recent call last):
File "/home/runner/.cache/rattler/cache/uv-cache/builds-v0/.tmpj7R2Pi/
lib/python3.12/site-packages/setuptools/_distutils/spawn.py", line 87,
in spawn
subprocess.check_call(cmd, env=_inject_macos_ver(env))
File "/home/runner/work/pymgcv/pymgcv/.pixi/envs/dev/lib/python3.12/
subprocess.py", line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/gcc', '-pthread',
'-B', '/home/runner/work/pymgcv/pymgcv/.pixi/envs/dev/compiler_compat',
'/tmp/tmp_pw_r_h8wjxaty/tmp/tmp_pw_r_h8wjxaty/test_pw_r.o', '-L/
home/runner/work/pymgcv/pymgcv/.pixi/envs/dev/lib', '-L/home/runner/
work/pymgcv/pymgcv/.pixi/envs/dev/lib/R/lib', '-L/home/runner/work/
pymgcv/pymgcv/.pixi/envs/dev/lib', '-lR', '-lpcre2-8', '-ldeflate',
'-llzma', '-lbz2', '-lz', '-lrt', '-ldl', '-lm', '-liconv', '-licuuc',
'-licui18n', '-o', '/tmp/tmp_pw_r_h8wjxaty/test_pw_r']' returned non-
zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/runner/.cache/rattler/cache/uv-cache/builds-v0/.tmpj7R2Pi/
lib/python3.12/site-packages/setuptools/_distutils/compilers/C/unix.py",
line 308, in link
self.spawn(linker + ld_args)
File "/home/runner/.cache/rattler/cache/uv-cache/builds-v0/.tmpj7R2Pi/
lib/python3.12/site-packages/setuptools/_distutils/compilers/C/base.py",
line 1158, in spawn
spawn(cmd, dry_run=self.dry_run, **kwargs)
File "/home/runner/.cache/rattler/cache/uv-cache/builds-v0/.tmpj7R2Pi/
lib/python3.12/site-packages/setuptools/_distutils/spawn.py", line 93,
in spawn
raise DistutilsExecError(
distutils.errors.DistutilsExecError: command '/usr/bin/gcc' failed with
exit code 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/home/runner/.cache/rattler/cache/uv-cache/builds-v0/.tmpj7R2Pi/
lib/python3.12/site-packages/setuptools/build_meta.py", line 334, in
get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/.cache/rattler/cache/uv-cache/builds-v0/.tmpj7R2Pi/
lib/python3.12/site-packages/setuptools/build_meta.py", line 304, in
_get_build_requires
self.run_setup()
File "/home/runner/.cache/rattler/cache/uv-cache/builds-v0/.tmpj7R2Pi/
lib/python3.12/site-packages/setuptools/build_meta.py", line 320, in
run_setup
exec(code, locals())
File "<string>", line 141, in <module>
File "<string>", line 121, in get_r_c_extension_status
File "<string>", line 82, in get_c_extension_status
File "/home/runner/.cache/rattler/cache/uv-cache/builds-v0/.tmpj7R2Pi/
lib/python3.12/site-packages/setuptools/_distutils/compilers/C/base.py",
line 841, in link_executable
self.link(
File "/home/runner/.cache/rattler/cache/uv-cache/builds-v0/.tmpj7R2Pi/
lib/python3.12/site-packages/setuptools/_distutils/compilers/C/unix.py",
line 3[10](https://github.com/danielward27/pymgcv/actions/runs/14417117507/job/40435039339#step:3:13), in link
raise LinkError(msg)
distutils.compilers.C.errors.LinkError: command '/usr/bin/gcc' failed
with exit code 1
hint: This usually indicates a problem with the package or the build
environment.
Error: The process '/home/runner/.pixi/bin/pixi' failed with exit code 1
Expected behavior
I would expect it to install successfully, as it does on my local machine.
Seems like a linker issue rather than pixi. Have you tried installing liblzma in your failing environment?
Seems like a linker issue rather than pixi. Have you tried installing liblzma in your failing environment?
You can do this by running pixi add liblzma
Thanks. I tried running: pixi add liblzma, with no luck and am getting the exact same error: The run is here https://github.com/danielward27/pymgcv/actions/runs/14473959906. Presumably some path variable is set wrong?
If this is an issue outside of pixi, feel free to close, but any hints are much appreciated too!
You can also try adding gcc from conda-forge. I often do pixi add compilers
@wolfv pointed out that you need to disable build isolation for the Python build backend to pick up the libraries.
Adding this to your pyproject.toml should do the trick:
[tool.pixi.pypi-options]
no-build-isolation = ["rpy2"]
You can read more at: https://pixi.sh/latest/reference/pixi_manifest/#no-build-isolation
Amazing, thanks everyone for the help. This works for me:
[project]
name = "pymgcv"
requires-python = ">= 3.11, <= 3.12"
version = "0.0.0"
dependencies = ["rpy2"]
[project.optional-dependencies]
dev = ["pytest"]
[build-system]
build-backend = "hatchling.build"
requires = ["hatchling"]
[tool.pytest.ini_options]
pythonpath = ["."]
# ------------------- pixi config --------------------#
[tool.pixi.dependencies]
r-base = "*"
setuptools = "*"
cffi = ">=1.17.1,<2" # https://github.com/rpy2/rpy2/issues/739
[tool.pixi.pypi-dependencies]
pymgcv = { path = ".", editable = true }
[tool.pixi.pypi-options]
no-build-isolation = ["rpy2"]
[tool.pixi.workspace]
channels = ["conda-forge"]
platforms = ["linux-64"]
[tool.pixi.environments]
dev = ["dev"]