scrapy-zyte-api icon indicating copy to clipboard operation
scrapy-zyte-api copied to clipboard

Switch auto field stats to an item pipeline

Open Gallaecio opened this issue 1 year ago • 2 comments

I was asked to support non-standard, custom items for auto field stats.

To achieve that, it seemed easier to switch the implementation to an item pipeline.

It also feels like this feature would make sense in scrapy-poet now. But I am not 100% sure, and I also would not want to move it until scrapy-poet gained add-on support.

To do:

  • [x] Merge and release https://github.com/zytedata/zyte-common-items/pull/102, and update setup.py and tox.ini accordingly.
  • [ ] Wait for @asadurski to test it out in a real project.

Gallaecio avatar Aug 26 '24 20:08 Gallaecio

Codecov Report

Attention: Patch coverage is 98.43750% with 1 line in your changes missing coverage. Please review.

Project coverage is 98.03%. Comparing base (a2061e8) to head (a5ffb08). Report is 37 commits behind head on main.

Files with missing lines Patch % Lines
scrapy_zyte_api/_poet_item_pipelines.py 98.33% 0 Missing and 1 partial :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #216      +/-   ##
==========================================
+ Coverage   97.86%   98.03%   +0.16%     
==========================================
  Files          14       16       +2     
  Lines        1590     1625      +35     
  Branches      337      342       +5     
==========================================
+ Hits         1556     1593      +37     
+ Misses         14       11       -3     
- Partials       20       21       +1     
Files with missing lines Coverage Δ
scrapy_zyte_api/addon.py 98.14% <100.00%> (+0.07%) :arrow_up:
scrapy_zyte_api/poet.py 100.00% <100.00%> (ø)
scrapy_zyte_api/providers.py 93.71% <100.00%> (-0.95%) :arrow_down:
scrapy_zyte_api/_poet_item_pipelines.py 98.33% <98.33%> (ø)

... and 1 file with indirect coverage changes

codecov[bot] avatar Aug 26 '24 21:08 codecov[bot]

The code looks good 👍 But I haven't checked how it works, and if it solves the issue :)

kmike avatar Aug 29 '24 18:08 kmike

Closing in favor of https://github.com/zytedata/zyte-common-items/pull/124.

Gallaecio avatar Dec 16 '24 15:12 Gallaecio