ggshield
ggshield copied to clipboard
GitGuardian
Stale dependency requirements are breaking the build on nixpkgs
Describe the bug
On nixpkgs-unstable, trying to build 1.39.0. We have to relax most of your python dependencies to do it. Here's what it looks like if we don't:
> Checking runtime dependencies for ggshield-1.39.0-py3-none-any.whl
> - platformdirs~=3.0.0 not satisfied by version 4.3.7
> - charset-normalizer~=3.1.0 not satisfied by version 3.4.1
> - cryptography~=43.0.1 not satisfied by version 44.0.2
> - marshmallow~=3.18.0 not satisfied by version 3.26.1
> - marshmallow-dataclass~=8.5.8 not satisfied by version 8.7.1
> - pygitguardian~=1.21.0 not satisfied by version 1.20.0
> - pyjwt~=2.6.0 not satisfied by version 2.10.1
> - python-dotenv~=0.21.0 not satisfied by version 1.1.0
> - rich~=12.5.1 not satisfied by version 14.0.0
> - typing-extensions~=4.12.2 not satisfied by version 4.13.0
> - urllib3~=2.2.2 not satisfied by version 2.3.0
If I relax the dependency requirements, this happens:
Running phase: pytestCheckPhase
Executing pytestCheckPhase
pytest flags: -m pytest --ignore-glob=tests/functional/ --ignore-glob=tests/unit/cmd/honeytoken --ignore-glob=tests/unit/cmd/scan/ --ignore-glob=tests/test_factories.py -k not\ \(test_cache_catches\)\ and\ not\ \(test_is_git_dir\)\ and\ not\ \(test_is_valid_git_commit_ref\)\ and\ not\ \(test_check_git_dir\)\ and\ not\ \(test_does_not_fail_if_cache\)\ and\ not\ \(test_create_files_from_paths\)\ and\ not\ \(test_file_decode_content\)\ and\ not\ \(test_file_is_longer_than_does_not_read_utf8_file\)\ and\ not\ \(test_file_is_longer_using_8bit_codec\)\ and\ not\ \(test_generate_files_from_paths\)\ and\ not\ \(test_get_file_sha_in_ref\)
============================= test session starts ==============================
platform darwin -- Python 3.11.11, pytest-8.3.5, pluggy-1.5.0
rootdir: /private/tmp/nix-build-ggshield-1.39.0.drv-0/source
configfile: pyproject.toml
testpaths: tests/
plugins: pyfakefs-5.8.0, factoryboy-2.6.1, Faker-37.1.0, mock-3.14.0, voluptuous-1.2.0, snapshottest-0.6.0, anyio-4.9.0
collected 999 items / 1 error / 23 deselected / 976 selected
==================================== ERRORS ====================================
___ ERROR collecting tests/unit/verticals/secret/output/test_text_output.py ____
tests/unit/verticals/secret/output/test_text_output.py:49: in <module>
Result.from_scan_result(
ggshield/verticals/secret/secret_scan_collection.py:187: in from_scan_result
secrets = [
ggshield/verticals/secret/secret_scan_collection.py:193: in <listcomp>
detector_name=policy_break.detector_name,
E AttributeError: 'PolicyBreak' object has no attribute 'detector_name'
=============================== warnings summary ===============================
../../../../nix/store/m4gnirj4cwf5xsiswia9aic1zgp4biiz-python3.11-marshmallow-3.26.1/lib/python3.11/site-packages/marshmallow/schema.py:129: 83 warnings
/nix/store/m4gnirj4cwf5xsiswia9aic1zgp4biiz-python3.11-marshmallow-3.26.1/lib/python3.11/site-packages/marshmallow/schema.py:129: RemovedInMarshmallow4Warning: The `ordered` `class Meta` option is deprecated. Field order is already preserved by default. Set `Schema.dict_class` to OrderedDict to maintain the previous behavior.
klass.opts = klass.OPTIONS_CLASS(meta, ordered=ordered)
../../../../nix/store/kdai2qcffbm03pv23nj5fngv658wswgs-python3.11-pygitguardian-1.20.0/lib/python3.11/site-packages/pygitguardian/models.py:532
/nix/store/kdai2qcffbm03pv23nj5fngv658wswgs-python3.11-pygitguardian-1.20.0/lib/python3.11/site-packages/pygitguardian/models.py:532: ChangedInMarshmallow4Warning: `Mapping` field should not be instantiated. Use `Dict` instead.
token = fields.Mapping(fields.String(), fields.String())
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
ERROR tests/unit/verticals/secret/output/test_text_output.py - AttributeError: 'PolicyBreak' object has no attribute 'detector_name'
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
================= 23 deselected, 84 warnings, 1 error in 0.82s =================
error: builder for '/nix/store/i7ix01gzzganjqxzggmp87mlr6kfp82z-ggshield-1.39.0.drv' failed with exit code 2;
last 25 log lines:
> collected 999 items / 1 error / 23 deselected / 976 selected
>
> ==================================== ERRORS ====================================
> ___ ERROR collecting tests/unit/verticals/secret/output/test_text_output.py ____
> tests/unit/verticals/secret/output/test_text_output.py:49: in <module>
> Result.from_scan_result(
> ggshield/verticals/secret/secret_scan_collection.py:187: in from_scan_result
> secrets = [
> ggshield/verticals/secret/secret_scan_collection.py:193: in <listcomp>
> detector_name=policy_break.detector_name,
> E AttributeError: 'PolicyBreak' object has no attribute 'detector_name'
> =============================== warnings summary ===============================
> ../../../../nix/store/m4gnirj4cwf5xsiswia9aic1zgp4biiz-python3.11-marshmallow-3.26.1/lib/python3.11/site-packages/marshmallow/schema.py:129: 83 warnings
> /nix/store/m4gnirj4cwf5xsiswia9aic1zgp4biiz-python3.11-marshmallow-3.26.1/lib/python3.11/site-packages/marshmallow/schema.py:129: RemovedInMarshmallow4Warning: The `ordered` `class Meta` option is deprecated. Field order is already preserved by default. Set `Schema.dict_class` to OrderedDict to maintain the previous behavior.
> klass.opts = klass.OPTIONS_CLASS(meta, ordered=ordered)
>
> ../../../../nix/store/kdai2qcffbm03pv23nj5fngv658wswgs-python3.11-pygitguardian-1.20.0/lib/python3.11/site-packages/pygitguardian/models.py:532
> /nix/store/kdai2qcffbm03pv23nj5fngv658wswgs-python3.11-pygitguardian-1.20.0/lib/python3.11/site-packages/pygitguardian/models.py:532: ChangedInMarshmallow4Warning: `Mapping` field should not be instantiated. Use `Dict` instead.
> token = fields.Mapping(fields.String(), fields.String())
>
> -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
> =========================== short test summary info ============================
> ERROR tests/unit/verticals/secret/output/test_text_output.py - AttributeError: 'PolicyBreak' object has no attribute 'detector_name'
> !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
> ================= 23 deselected, 84 warnings, 1 error in 0.82s =================
This error changes when updating to pygitguardian 1.21.0 per the requirements:
error: builder for '/nix/store/i8aapckiic2r77xh06s4mhii0jciaf4v-ggshield-1.38.1.drv' failed with exit code 2;
last 25 log lines:
> <frozen importlib._bootstrap>:690: in _load_unlocked
> ???
> /nix/store/h3piq8dh96lwvvawabf90frdwl6yabhn-python3.11-pytest-8.3.5/lib/python3.11/site-packages/_pytest/assertion/rewrite.py:185: in exec_module
> exec(co, module.__dict__)
> tests/unit/conftest.py:60: in <module>
> _MULTIPLE_SECRETS_SCAN_RESULT = ScanResult.SCHEMA.load(
> /nix/store/m4gnirj4cwf5xsiswia9aic1zgp4biiz-python3.11-marshmallow-3.26.1/lib/python3.11/site-packages/marshmallow/schema.py:792: in load
> return self._do_load(
> /nix/store/m4gnirj4cwf5xsiswia9aic1zgp4biiz-python3.11-marshmallow-3.26.1/lib/python3.11/site-packages/marshmallow/schema.py:999: in _do_load
> raise exc
> E marshmallow.exceptions.ValidationError: {'policy_breaks': {0: {'detector_name': ['Missing data for required field.'], 'detector_group_name': ['Missing data for required field.']}}}
> =============================== warnings summary ===============================
> ../../../../nix/store/m4gnirj4cwf5xsiswia9aic1zgp4biiz-python3.11-marshmallow-3.26.1/lib/python3.11/site-packages/marshmallow/schema.py:129: 76 warnings
> /nix/store/m4gnirj4cwf5xsiswia9aic1zgp4biiz-python3.11-marshmallow-3.26.1/lib/python3.11/site-packages/marshmallow/schema.py:129: RemovedInMarshmallow4Warning: The `ordered` `class Meta` option is deprecated. Field order is already preserved by default. Set `Schema.dict_class` to OrderedDict to maintain the previous behavior.
> klass.opts = klass.OPTIONS_CLASS(meta, ordered=ordered)
>
> ../../../../nix/store/v6kn4kfz1p79ap5c9yd1iry4znd5hbds-python3.11-pygitguardian-1.21.0/lib/python3.11/site-packages/pygitguardian/models.py:543
> /nix/store/v6kn4kfz1p79ap5c9yd1iry4znd5hbds-python3.11-pygitguardian-1.21.0/lib/python3.11/site-packages/pygitguardian/models.py:543: ChangedInMarshmallow4Warning: `Mapping` field should not be instantiated. Use `Dict` instead.
> token = fields.Mapping(fields.String(), fields.String())
>
> -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
> =========================== short test summary info ============================
> ERROR tests/unit - marshmallow.exceptions.ValidationError: {'policy_breaks': {0: {'detector_na...
> !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
> ======================== 77 warnings, 1 error in 1.74s =========================
It turns out that bumping to [email protected] resolves this, but it would still be a good idea to test with modern dependencies.
Hi, sorry for the late answer. Indeed we pin our dependencies quite aggressively.
We prefer doing it this way because some packages do not follow semver even if we could think they do (recent example: click) and some intent to follow semver but sometimes fail to do so. Only allowing patch upgrades provides a better protection for that kind of problems.
As you noticed, updating py-gitguardian fixes your issue.
