oltarasenko
oltarasenko
Yes, I think it should be fine to do that. However, if the configuration is not set the pipeline should probably default to the CrawlyUI..
Invite sent.
Yes, it's absolutely true. We had plans both for Request/Items storages to be able to share and recover states... however it turned out that for now, we did not have...
@s0kil I think @Ziinc gave a good answer, pipeline is a good way to go! Otherwise, in my own projects, I am downloading media from the `parse_item` callback directly. Crawly...
No, it does not block the Crawly itself, just one worker which is downloading something, but all other workers are operational. (Comparing it with Scrapy, where non-reactor based downloads will...
I will close this one, as it's been open for years, and no one has had time or need to lead the work.
Hard to say. I did not have a chance to explore these two tools. In some of my previous projects, phantom js was used for browser rendering, but now it...
I like the idea!
Hey @aej I have tried to address this idea in the: https://github.com/elixir-crawly/crawly/pull/265
Yes sure. There is no reason to keep it unreleased. let me prepare the release early this week