Switch auto field stats to an item pipeline
I was asked to support non-standard, custom items for auto field stats.
To achieve that, it seemed easier to switch the implementation to an item pipeline.
It also feels like this feature would make sense in scrapy-poet now. But I am not 100% sure, and I also would not want to move it until scrapy-poet gained add-on support.
To do:
- [x] Merge and release https://github.com/zytedata/zyte-common-items/pull/102, and update
setup.pyandtox.iniaccordingly. - [ ] Wait for @asadurski to test it out in a real project.
Codecov Report
Attention: Patch coverage is 98.43750% with 1 line in your changes missing coverage. Please review.
Project coverage is 98.03%. Comparing base (
a2061e8) to head (a5ffb08). Report is 37 commits behind head on main.
| Files with missing lines | Patch % | Lines |
|---|---|---|
| scrapy_zyte_api/_poet_item_pipelines.py | 98.33% | 0 Missing and 1 partial :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #216 +/- ##
==========================================
+ Coverage 97.86% 98.03% +0.16%
==========================================
Files 14 16 +2
Lines 1590 1625 +35
Branches 337 342 +5
==========================================
+ Hits 1556 1593 +37
+ Misses 14 11 -3
- Partials 20 21 +1
| Files with missing lines | Coverage Δ | |
|---|---|---|
| scrapy_zyte_api/addon.py | 98.14% <100.00%> (+0.07%) |
:arrow_up: |
| scrapy_zyte_api/poet.py | 100.00% <100.00%> (ø) |
|
| scrapy_zyte_api/providers.py | 93.71% <100.00%> (-0.95%) |
:arrow_down: |
| scrapy_zyte_api/_poet_item_pipelines.py | 98.33% <98.33%> (ø) |
The code looks good 👍 But I haven't checked how it works, and if it solves the issue :)
Closing in favor of https://github.com/zytedata/zyte-common-items/pull/124.