oltarasenko
oltarasenko
@dogweather Looks like it's not enough. Probably error messages (even debug), make people think that Crawly is broken and no spiders will run :(. So I think it's better to...
I think the debug message we have now, looks quite ok: `12:57:29.967 [info] No spiders found to auto-load: %MatchError{term: {:error, :enoent}}`
Hey @assertnotnull the code above seem to work fine for me: ``` iex(2)> 10:45:07.962 [warning] Description: 'Server authenticity is not verified since certificate path validation is not enabled' Reason: 'The...
Hey @assertnotnull the code above seem to work fine for me: ``` iex(2)> 10:45:07.962 [warning] Description: 'Server authenticity is not verified since certificate path validation is not enabled' Reason: 'The...
Strange. @zongwu233, as I see that code was added 6 months ago. In any case, I am preparing the 0.15.0, so hopefully it will disappear soon.
Strange. @zongwu233, as I see that code was added 6 months ago. In any case, I am preparing the 0.15.0, so hopefully it will disappear soon.
I think it makes sense to introduce triggers instead of the count estimate. Some related articles to read: http://www.varlena.com/GeneralBits/49.php http://www.varlena.com/GeneralBits/120.php
@pablohoffman I have fixed PR notes.
@qrilka @pablohoffman I have added a flag which controls if random ua headers rotation is used or not. Please review and merge!
@pablohoffman I have reduced the variable name. But looks like @qrilka still have questions to the PR, so lets not merge it just now :(