oltarasenko
oltarasenko
Yes, sorry: https://www.yooli.com/secure/login/ And the html is in gist: https://gist.github.com/oltarasenko/2b136b2e3791c0d27f75
Just in case, could you make an update to 0.14.0. For me it looks like this: ``` Generated quickstart app 11:03:59.519 [debug] Starting data storage 11:03:59.524 [debug] Starting the manager...
Actually I think that it's possible to get the stats from the DataStorage: e.g. `Crawly.DataStorage.stats(spider_name)`. https://github.com/oltarasenko/crawly/blob/master/lib/crawly/manager.ex#L83
@Ziinc I am thinking of making a major release of Crawly (aka v1.0.0). I think after a year of development and releases it's time to do that (I am seeing...
@Ziinc yes we need to aim to get 1.0.0 release. It's a bit hard to push Crawly into production for larger products atm. The fact that we don't have a...
@Ziinc we also did not use Selenium for real scraping back in time when I was working with scrapy. My personal vision of it: 1. It was not stable enough...
@Ziinc maybe you're right. I don't have a full picture of it yet. I would try to build a prototype and see what is required from the production usages. From...
Hey @EdmondFrank . It's not quite clear if this approach is going to solve the issue. But still, what is the problem of having just the closespider timeout
Yes, I would agree I've never planned to use a full URL here, however, decided to keep it this way, as it was faster at that point (I am talking...
Hey. As far as I see, it failed on --check-formatted step. Could you please format the code so I can merge it? @jack-s9