KeyError when pyproject.toml dependencies are dynamically read from requirements.txt

[fofoni]

Describe the bug

I usually have the following structure in my projects:

# pyproject.toml
[build-system]
requires = ["setuptools>=67.6.0"]
build-backend = "setuptools.build_meta"
[project]
dynamic = ["dependencies"]
# no `dependencies` key
[tool.setuptools.dynamic]
dependencies = { file = ["requirements.txt"] }

And then the projects requirements are listed in the requirements.txt file. This is useful because IDEs will usually syntax-highlight PEP 440 syntax in requirements files, but not in strings inside TOML tables.

When I run deptry . with such a project, it crashes with the following traceback:

Traceback (most recent call last):
  File ".../bin/deptry", line 8, in <module>
    sys.exit(deptry())
  File ".../python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
[...]
  File ".../python3.10/site-packages/deptry/core.py", line 63, in run
    dependencies_extract = self._get_dependencies(dependency_management_format)
  File ".../python3.10/site-packages/deptry/core.py", line 142, in _get_dependencies
    return PEP621DependencyGetter(self.config, self.package_module_name_map).get()
  File ".../python3.10/site-packages/deptry/dependency_getter/pep_621.py", line 41, in get
    dependencies = [*self._get_dependencies(), *itertools.chain(*self._get_optional_dependencies().values())]
  File ".../python3.10/site-packages/deptry/dependency_getter/pep_621.py", line 48, in _get_dependencies
    dependency_strings: list[str] = pyproject_data["project"]["dependencies"]
KeyError: 'dependencies'

To Reproduce

Using a pyproject.toml file as described above (with no dependencies key inside the top-level project table, but mentioning "dependencies" as a dynamic metadata), inside a Python 3.10 virtual environment, run:

pip install -e .
pip install deptry
deptry .

Expected behavior

Either:

• Support for the the tool.setuptools.dynamic.dependencies.file idiom (see https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html#dynamic-metadata); or
• A clear error message saying that this idiom is not supported.

System:

• OS: Ubuntu 20.04 inside WSL (Windows 10)
• Language Version: Python 3.10.12
• Poetry version: [not using poetry or any other packaging framework]

[mkniewallner]

Interesting, didn't know that it was possible to do that with setuptools. This is possibly something we could implement.

Unfortunately, until this is implemented, there is no escape hatch to force deptry to use requirements.txt, even when using --requirements-txt requirements.txt, since it will still use PEP 621 metadata from pyproject.toml if a project key is found (see here and here). It might also be something we want to improve in the future.

[homeworkprod]

Same issue here: pyproject.toml without project.dependencies key and with a requirements.txt leads to the stack trace posted above by @fofoni.

I'd say this still fits the use case stated as supported ("Projects that use a requirements.txt file according to the pip standards").

[Pushkal-G]

Hello, I am interested in working on this issue. Since I am new to project, I will appreciate if I can get a bit more guidance on what changes are supposed to be made and what is the expected behaviour?

[alfechner]

We have the same issue for all our Python projects in our organisation.

[mkniewallner]

There is ongoing work to support this in https://github.com/fpgmaas/deptry/pull/881, if anyone wants to try it out and report back on whether this works.