hatch
hatch copied to clipboard
pypa
Feature: Support dynamic `optional-dependencies` in `hatch env`
Currently when project defines "optional-dependencies" as dynamic property, hatch cannot use such dynamic property in env definition.
Example is Apache Airflow. After we turned optional-dependencies to dynamic field, we had to remove devel from features:
https://github.com/apache/airflow/blob/main/pyproject.toml#L159
[tool.hatch.envs.default]
python = "3.8"
platforms = ["linux", "macos"]
description = "Default environment with Python 3.8 for maximum compatibility"
features = []
When you try to use devel feature:
[tool.hatch.envs.default]
python = "3.8"
platforms = ["linux", "macos"]
description = "Default environment with Python 3.8 for maximum compatibility"
features = ["devel"] # <-- add `devel` feature here
The hatch env command almost immediately fails with:
ValueError: Feature `devel` of field `tool.hatch.envs.default.features` is not defined in field `project.optional-dependencies`
Can you please try master to see if this fixes it? https://github.com/pypa/hatch/pull/1387
Tried, seems it still does not work for airflow:
-
Checked out latest airflow (https://github.com/apache/airflow)
-
Installed latest hatch
pip install "hatch @ git+https://github.com/pypa/hatch" -
Double check:
[jarek:~/code/airflow] [apache-airflow] main+ ± hatch --version
Hatch, version 1.9.2.dev90
- Modify pyproject.toml:
[tool.hatch.envs.default]
python = "3.8"
platforms = ["linux", "macos"]
description = "Default environment with Python 3.8 for maximum compatibility"
features = []
to:
[tool.hatch.envs.default]
python = "3.8"
platforms = ["linux", "macos"]
description = "Default environment with Python 3.8 for maximum compatibility"
features = ["devel"]
hatch env create
/home/jarek/.pyenv/versions/3.10.13/lib/python3.10/site-packages/hatch/cli/__init__.py:221 in │
│ main │
│ │
│ 218 │
│ 219 def main(): # no cov │
│ 220 │ try: │
│ ❱ 221 │ │ hatch(prog_name='hatch', windows_expand_args=False) │
│ 222 │ except Exception: # noqa: BLE001 │
│ 223 │ │ import sys │
│ 224 │
│ │
│ /home/jarek/.pyenv/versions/3.10.13/lib/python3.10/site-packages/click/core.py:1157 in __call__ │
│ │
│ /home/jarek/.pyenv/versions/3.10.13/lib/python3.10/site-packages/click/core.py:1078 in main │
│ │
│ /home/jarek/.pyenv/versions/3.10.13/lib/python3.10/site-packages/click/core.py:1688 in invoke │
│ │
│ /home/jarek/.pyenv/versions/3.10.13/lib/python3.10/site-packages/click/core.py:1688 in invoke │
│ │
│ /home/jarek/.pyenv/versions/3.10.13/lib/python3.10/site-packages/click/core.py:1434 in invoke │
│ │
│ /home/jarek/.pyenv/versions/3.10.13/lib/python3.10/site-packages/click/core.py:783 in invoke │
│ │
│ /home/jarek/.pyenv/versions/3.10.13/lib/python3.10/site-packages/click/decorators.py:45 in │
│ new_func │
│ │
│ /home/jarek/.pyenv/versions/3.10.13/lib/python3.10/site-packages/hatch/cli/env/create.py:38 in │
│ create │
│ │
│ 35 │ │ │ │
│ 36 │ │ │ app.abort(f'Environment `{env}` is incompatible: {e}') │
│ 37 │ │ │
│ ❱ 38 │ │ app.prepare_environment(environment) │
│ 39 │ │
│ 40 │ if incompatible: │
│ 41 │ │ num_incompatible = len(incompatible) │
│ │
│ /home/jarek/.pyenv/versions/3.10.13/lib/python3.10/site-packages/hatch/cli/application.py:125 in │
│ prepare_environment │
│ │
│ 122 │ │ │ │ │
│ 123 │ │ │ │ with environment.app_status_project_installation(): │
│ 124 │ │ │ │ │ if environment.dev_mode: │
│ ❱ 125 │ │ │ │ │ │ environment.install_project_dev_mode() │
│ 126 │ │ │ │ │ else: │
│ 127 │ │ │ │ │ │ environment.install_project() │
│ 128 │
│ │
│ /home/jarek/.pyenv/versions/3.10.13/lib/python3.10/site-packages/hatch/env/virtual.py:164 in │
│ install_project_dev_mode │
│ │
│ 161 │ def install_project_dev_mode(self): │
│ 162 │ │ with self.safe_activation(): │
│ 163 │ │ │ self.platform.check_command( │
│ ❱ 164 │ │ │ │ self.construct_pip_install_command(['--editable', self.apply_features(st │
│ 165 │ │ │ ) │
│ 166 │ │
│ 167 │ def dependencies_in_sync(self): │
│ │
│ /home/jarek/.pyenv/versions/3.10.13/lib/python3.10/site-packages/hatch/env/plugin/interface.py:9 │
│ 19 in apply_features │
│ │
│ 916 │ │ A convenience method that applies any user defined [features](../../config/envir │
│ 917 │ │ to the given requirement. │
│ 918 │ │ """ │
│ ❱ 919 │ │ if self.features: │
│ 920 │ │ │ features = ','.join(self.features) │
│ 921 │ │ │ return f'{requirement}[{features}]' │
│ 922 │
│ │
│ /home/jarek/.pyenv/versions/3.10.13/lib/python3.10/site-packages/hatch/env/plugin/interface.py:4 │
│ 43 in features │
│ │
│ 440 │ │ │ │ │ │ f'Feature `{normalized_feature}` of field `tool.hatch.envs.{self │
│ 441 │ │ │ │ │ │ f'defined in field `project.optional-dependencies`' │
│ 442 │ │ │ │ │ ) │
│ ❱ 443 │ │ │ │ │ raise ValueError(message) │
│ 444 │ │ │ │ │
│ 445 │ │ │ │ all_features.add(normalized_feature) │
│ 446 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: Feature `devel` of field `tool.hatch.envs.default.features` is not defined in field `project.optional-dependencies`
- double check that installation with
develextra works with pip in editable mode:
Change back pyproject.toml tpo features=[ ]
hatch env create -> works and install airflow in editable mode without devel deps (for example without pytest)
> hatch shell
> python -c 'import pytest'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'pytest'
Re-install airflow in --editable mode and with devel extras:
pip install -e ".[devel]"
Successfully installed SecretStorage-3.3.3 aiofiles-23.2.1 aioresponses-0.7.6 amqp-5.2.0 apache-airflow-2.10.0.dev0 asttokens-2.4.1 backcall-0.2.0 backports.tarfile-1.0.0 beautifulsoup4-4.12.3 billiard-4.2.0 black-24.4.0 cachetools-5.3.3 celery-5.3.6 cfgv-3.4.0 click-didyoumean-0.3.1 click-plugins-1.1.1 click-repl-0.3.0 coverage-7.4.4 decorator-5.1.1 distlib-0.3.8 duckdb-0.10.1 editables-0.5 execnet-2.1.1 executing-2.0.1 filelock-3.13.4 flower-2.0.1 gitdb-4.0.11 gitpython-3.1.43 google-auth-2.29.0 hatch-1.9.4 hatchling-1.21.1 humanize-4.9.0 hyperlink-21.0.0 icdiff-2.0.7 identify-2.5.35 incremental-22.10.0 iniconfig-2.0.0 ipdb-0.13.13 ipython-8.12.3 jaraco.classes-3.4.0 jaraco.context-5.3.0 jaraco.functools-4.0.0 jedi-0.19.1 jeepney-0.8.0 keyring-25.1.0 kombu-5.3.7 kubernetes-29.0.0 kubernetes_asyncio-29.0.0 matplotlib-inline-0.1.7 mypy-1.9.0 mypy-extensions-1.0.0 nh3-0.2.17 nodeenv-1.8.0 oauthlib-3.2.2 parso-0.8.4 pexpect-4.9.0 pickleshare-0.7.5 pip-24.0 pipdeptree-2.18.1 pkginfo-1.10.0 platformdirs-4.2.0 pprintpp-0.4.0 pre-commit-3.5.0 prometheus-client-0.20.0 prompt-toolkit-3.0.43 ptyprocess-0.7.0 pure-eval-0.2.2 pyasn1-0.6.0 pyasn1-modules-0.4.0 pygithub-2.3.0 pynacl-1.5.0 pytest-7.4.4 pytest-asyncio-0.23.6 pytest-cov-5.0.0 pytest-custom-exit-code-0.3.0 pytest-icdiff-0.9 pytest-instafail-0.5.0 pytest-mock-3.14.0 pytest-rerunfailures-14.0 pytest-timeouts-1.2.1 pytest-xdist-3.5.0 readme-renderer-43.0 redis-5.0.3 requests-oauthlib-2.0.0 requests_mock-1.12.1 restructuredtext-lint-1.4.0 rfc3986-2.0.0 rich-click-1.7.4 rsa-4.9 ruff-0.3.3 semver-3.0.2 shellingham-1.5.4 smmap-5.0.1 soupsieve-2.5 stack-data-0.6.3 tomli-2.0.1 tomli-w-1.0.0 tomlkit-0.12.4 tornado-6.4 towncrier-23.11.0 traitlets-5.14.2 trove-classifiers-2024.4.10 twine-5.0.0 types-Deprecated-1.2.9.20240311 types-Markdown-3.6.0.20240316 types-PyMySQL-1.1.0.1 types-PyYAML-6.0.12.20240311 types-aiofiles-23.2.0.20240403 types-certifi-2021.10.8.3 types-croniter-2.0.0.20240321 types-docutils-0.20.0.20240406 types-paramiko-3.4.0.20240311 types-protobuf-4.25.0.20240410 types-pyOpenSSL-24.0.0.20240311 types-python-dateutil-2.9.0.20240316 types-python-slugify-8.0.2.20240310 types-pytz-2024.1.0.20240203 types-redis-4.6.0.20240409 types-requests-2.31.0.20240406 types-setuptools-69.5.0.20240415 types-tabulate-0.9.0.20240106 types-termcolor-1.1.6.2 types-toml-0.10.8.20240310 userpath-1.9.2 vine-5.1.0 virtualenv-20.25.1 wcwidth-0.2.13 websocket-client-1.7.0 wheel-0.43.0 yamllint-1.35.1 zstandard-0.22.0