scrapy icon indicating copy to clipboard operation
scrapy copied to clipboard

fix duplicate keys in settings.getwithbase() (#6912)

Open Aearsears opened this issue 4 months ago • 3 comments

Resolves #6912

  • Fixes duplicate keys in base settings and in user defined settings by removing the import path string. logs a warning to the user
  • add test for duplicate keys
  • update dev dependencies in pyproject.toml
  • update docs mentioning ambiguous behaviour if users specify both object and import path
  • update docs about how to install dev dependencies

initially I thought of using load_object to check but that seems too costly if the default settings contain the entire list of middlewares, so I opt for checking the import path strings instead

Aearsears avatar Aug 09 '25 03:08 Aearsears

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 90.69%. Comparing base (a0b766f) to head (a1c8e1b). :white_check_mark: All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7000      +/-   ##
==========================================
+ Coverage   90.68%   90.69%   +0.01%     
==========================================
  Files         164      164              
  Lines       12642    12657      +15     
  Branches     1643     1649       +6     
==========================================
+ Hits        11464    11479      +15     
  Misses        891      891              
  Partials      287      287              
Files with missing lines Coverage Δ
scrapy/settings/__init__.py 98.03% <100.00%> (+0.12%) :arrow_up:

codecov[bot] avatar Aug 09 '25 08:08 codecov[bot]

the test that is failing is TestSendCatchLogAsyncAsyncio in tests\test_utils_signal.py . When the unit tests ran last night, this test passed. this morning, i added only type annotations to sets and rebased my changes onto master. The rest of the Ubuntu python tests are passing. I also ran TestSendCatchLogAsyncAsyncio locally and its passing.

I'm not entirely sure what is causing this test to fail. could i please get some more context? could it be a timing issue?

Aearsears avatar Aug 09 '25 16:08 Aearsears

That's a rare failure likely introduced by changes in #6997, you should ignore it

wRAR avatar Aug 09 '25 16:08 wRAR