Adrián Chaves comments

Results 860 comments of


                                            Adrián Chaves

Use of `functools.lru_cache` or `functools.cache` on methods can lead to memory leaks

Since we use this to convert expressions that are usually only a few and defined at development time, in code, and not at run time, and we limit the sizes...

Recheck (and re-fix?) open_spider exception handling

> it's unclear to me what's the intended behavior +1

Engine hangs if _schedule_request() raised an exception

I think https://github.com/scrapy/scrapy/pull/6715 addresses this (one of the changes is about good handling of unhandled exceptions from user-customizable APIs, mainly scheduler methods), maybe we can extract the related changes into...

Fix typo in cmdline.py comment: 'a argument' -> 'an argument'

Could you revert https://github.com/scrapy/scrapy/pull/6875/commits/d66f649a29270beceeceaebb048d7735593016bd? I don’t think we need tests for this change.

Fix typo in cmdline.py comment: 'a argument' -> 'an argument'

Hmm… Those failures are completely unrelated. I wonder what’s triggering them, though.

Fix typo in cmdline.py comment: 'a argument' -> 'an argument'

Thanks!

Added error message in stats if exception is thrown in start_requests of spider

I’m now thinking that maybe we should only add the exception class name to the stats. I’m guessing the stat breadcrumbs are not meant to have `/` characters, and we...

DummyResponse

The behavior here is as designed, Zyte API params for provider dependencies can only be altered with the `zyte_api_provider` request metadata key or the `ZYTE_API_PROVIDER_PARAMS` setting. However, I wonder if...

Strange parser error: search_dates parses "2010 Year" to a date with year of 4033

This is because year is interpreted the same as years, and “2010 years” is interpreted as “2010 years later“. Maybe we could make it so that if it is year,...

Strange parser error: search_dates parses "2010 Year" to a date with year of 4033

The code base is relatively complex, and I don’t think this case is user configurable at the moment.