UltraNest
UltraNest copied to clipboard
Published
20 hours ago •
JohannesBuchner
JohannesBuchner
Ultranest attempts a requirement import during installation
- UltraNest version: 3.3.3
- Python version: 3.9+
- Operating System: Mac OS Big Sur
Description
During installation Ultranest tries to import Cython. Pip installations bundle gathering dependencies and then at a later stage they install all the dependencies. The bundles aren't actually installed yet. So when Ultranest tries to reference Cython during the installation process in setup.py, it's unavailable.
someorg:EXOTIC jpl-jengelke$ pip install -r requirements.txt
Ignoring importlib-metadata: markers 'python_version <= "3.7"' don't match your environment
Ignoring dynesty: markers 'platform_system == "Windows"' don't match your environment
Collecting wheel
Downloading wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Requirement already satisfied: setuptools in /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages (from -r requirements.txt (line 3)) (60.9.3)
Collecting python_version
Downloading python_version-0.0.2-py2.py3-none-any.whl (3.4 kB)
Collecting astroalign~=2.4
Downloading astroalign-2.4.1.tar.gz (1.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 10.8 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Collecting astropy>=4.3
Downloading astropy-5.0.2-cp39-cp39-macosx_10_9_x86_64.whl (6.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.9/6.9 MB 12.4 MB/s eta 0:00:00
Collecting astroquery~=0.4
Downloading astroquery-0.4.5-py3-none-any.whl (4.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 15.3 MB/s eta 0:00:00
Collecting barycorrpy~=0.4
Downloading barycorrpy-0.4.4-py3-none-any.whl (9.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.6/9.6 MB 14.5 MB/s eta 0:00:00
Collecting cython~=0.29.26
Downloading Cython-0.29.28-py2.py3-none-any.whl (983 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 983.8/983.8 KB 8.5 MB/s eta 0:00:00
Collecting holoviews~=1.14
Downloading holoviews-1.14.8-py2.py3-none-any.whl (4.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.3/4.3 MB 15.2 MB/s eta 0:00:00
Collecting LDTk~=1.7
Downloading LDTk-1.7.0-py3-none-any.whl (37 kB)
Collecting lmfit~=1.0
Downloading lmfit-1.0.3.tar.gz (292 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 292.5/292.5 KB 6.4 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Collecting matplotlib>=3.4
Downloading matplotlib-3.5.1-cp39-cp39-macosx_10_9_x86_64.whl (7.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.3/7.3 MB 3.7 MB/s eta 0:00:00
Collecting numpy~=1.21
Downloading numpy-1.22.3-cp39-cp39-macosx_10_14_x86_64.whl (17.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.6/17.6 MB 14.3 MB/s eta 0:00:00
Collecting pandas~=1.3
Downloading pandas-1.4.1-cp39-cp39-macosx_10_9_x86_64.whl (11.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.5/11.5 MB 15.2 MB/s eta 0:00:00
Collecting panel~=0.12
Downloading panel-0.12.6-py2.py3-none-any.whl (12.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.9/12.9 MB 15.6 MB/s eta 0:00:00
Collecting photutils>=0.7
Downloading photutils-1.3.0-cp39-cp39-macosx_10_9_x86_64.whl (559 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 559.1/559.1 KB 8.2 MB/s eta 0:00:00
Collecting python_dateutil~=2.8
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 KB 5.6 MB/s eta 0:00:00
Collecting pyvo~=1.2
Downloading pyvo-1.3-py3-none-any.whl (829 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 829.6/829.6 KB 8.5 MB/s eta 0:00:00
Collecting requests~=2.27
Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.1/63.1 KB 2.0 MB/s eta 0:00:00
Collecting scipy~=1.7
Downloading scipy-1.8.0-cp39-cp39-macosx_12_0_universal2.macosx_10_9_x86_64.whl (55.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 55.6/55.6 MB 13.2 MB/s eta 0:00:00
Collecting scikit-image~=0.18
Downloading scikit_image-0.19.2-cp39-cp39-macosx_10_13_x86_64.whl (13.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.3/13.3 MB 15.7 MB/s eta 0:00:00
Collecting tenacity~=8.0
Downloading tenacity-8.0.1-py3-none-any.whl (24 kB)
Collecting ultranest~=3.3.3
Downloading ultranest-3.3.3.tar.gz (21.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.5/21.5 MB 14.3 MB/s eta 0:00:00
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/private/var/folders/0_/0mssd1656q78c47k0_yk7sn00000gp/T/pip-install-zzyhdkbo/ultranest_45cde6e84d9d4627ab12dbddb6315fea/setup.py", line 9, in <module>
from Cython.Build import cythonize
ModuleNotFoundError: No module named 'Cython'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
WARNING: You are using pip version 22.0.3; however, version 22.0.4 is available.
You should consider upgrading via the '/Library/Frameworks/Python.framework/Versions/3.9/bin/python3 -m pip install --upgrade pip' command.
What I Did
Workaround is to install Cython first, so Ultranest can use it on a second install pass. We made two requirements files to call.
Yes, this is a known limitation of pip.
See https://github.com/JohannesBuchner/UltraNest/issues/26. Could potentially be resolved with pyproject.toml.
I'd be interested in seeing it resolved if you have the time to look into creating such a pyproject.toml file for ultranest.
Not sure if there is a way to solve this. I found: https://sadafnoor.me/2015/01/07/how-to-automate-numpy-installation-in-your-project-using-setuptool/ but it's unclear if this invokes the limitation.
Thank you. I will see if I have time to implement the TOML solution.
The last round of releases seems to have triggered this issue for us downstream as we use Ultranest within our testsuite. I'm not sure why it would be a problem with the April 6th release and not earlier though. Perhaps pip just decided to install stuff in a different order.
I don't see a PR ready to go atm, so I'll look into one for Ultrnest. @JohannesBuchner Would it be possible to do a small point release after a proposed PR is merged to make sure this works as intended (and to resolve downstream issues)?
Ok, I believe this is resolved by https://github.com/JohannesBuchner/UltraNest/pull/59
The error we see with 3.4.2 of ultranest is as follows
Collecting ultranest
Downloading ultranest-3.4.2.tar.gz (21.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.8/21.8 MB 76.6 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
WARNING: Missing build requirements in pyproject.toml for ultranest from https://files.pythonhosted.org/packages/12/50/71eb4b4f259c524fc5783300db4ffc2447573874d18015deef95bced891a/ultranest-3.4.2.tar.gz#sha256=b3872217c648a4e98932b94935a554f9b55a5fe225709f60336677a887ecb1ab (from -r companion.txt (line 17)).
WARNING: The project does not specify a build backend, and pip cannot fall back to setuptools without 'setuptools>=40.8.0' and 'wheel'.
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
ERROR: Exception:
Traceback (most recent call last):
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
status = run_func(*args)
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
return func(self, options, args)
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 339, in run
requirement_set = resolver.resolve(
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 94, in resolve
result = self._result = resolver.resolve(
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 348, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
if not criterion.candidates:
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
return bool(self._sequence)
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
return any(self)
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 1[43](https://github.com/ahnitz/pycbc/runs/5868694991?check_suite_focus=true#step:6:43), in <genexpr>
return (c for c in iterator if id(c) not in self._incompatible_ids)
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
candidate = func()
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 215, in _make_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 288, in __init__
super().__init__(
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 158, in __init__
self.dist = self._prepare()
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 227, in _prepare
dist = self._prepare_distribution()
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 299, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 487, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 556, in _prepare_linked_requirement
dist = _get_prepared_distribution(
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 58, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(finder, build_isolation)
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line [45](https://github.com/ahnitz/pycbc/runs/5868694991?check_suite_focus=true#step:6:45), in prepare_distribution_metadata
self._install_build_reqs(finder)
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line 104, in _install_build_reqs
build_reqs = self._get_build_requires_wheel()
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line 81, in _get_build_requires_wheel
return backend.get_requires_for_build_wheel()
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_vendor/pep[51](https://github.com/ahnitz/pycbc/runs/5868694991?check_suite_focus=true#step:6:51)7/wrappers.py", line 1[72](https://github.com/ahnitz/pycbc/runs/5868694991?check_suite_focus=true#step:6:72), in get_requires_for_build_wheel
return self._call_hook('get_requires_for_build_wheel', {
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_vendor/pep517/wrappers.py", line 332, in _call_hook
raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 89, in _build_backend
obj = import_module(mod_path)
File "/home/runner/work/pycbc/pycbc/.tox/py-inference/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 9[84](https://github.com/ahnitz/pycbc/runs/5868694991?check_suite_focus=true#step:6:84), in _find_and_load_unlocked
ModuleNotFoundError: No module named 'setuptools'
OK, I added commit 46abf66, and should release a new version to pypi.
ultranest 3.4.3 is on pypi
The conda-forge issues also seems to be solved soon.