[434] Allow to filter jobs in ZyteJobsComparisonMonitor by close_reason

[shafiq-muhammad]

Adds limit for nested dict stats computation

Added a new setting SPIDERMON_JOBS_COMPARISON_CLOSE_REASONS to allow ZyteJobsComparisonMonitor to filter by the ScrapyCloud jobs with their close_reason stat.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 79.54%. Comparing base (6fe8928) to head (7260d08).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #440      +/-   ##
==========================================
+ Coverage   79.51%   79.54%   +0.03%     
==========================================
  Files          76       76              
  Lines        3237     3242       +5     
  Branches      537      539       +2     
==========================================
+ Hits         2574     2579       +5     
  Misses        593      593              
  Partials       70       70              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[aaneto]

Just a quick FYI @shafiq-muhammad @curita, I believe the pipeline is failing due to a regression from pytest 8.1.1 to 8.2.0.

My validation was:

• Creating an empty .env for the project and running pip install tox and tox -e base
• Noticing the errors, and setting pytest==8.1.1 on tox.ini
• Running tox -e base again, with success this time.

Hope this helps (and is correct regardless of environment).

[VMRuiz]

Yes, it looks like Pytest 8.2.0 release has caused a lot of troubles according to their issues page. @shafiq-muhammad , Could you please pin pytest to 8.1.1 until they release a new version?