Scraper

Scraper copied to clipboard

Is there a way of being able to scrape a site that requires a login?

lewisjenkins

c.f. https://github.com/voku/simple_html_dom or, just remove that driver in favor of more modern options (Goute, Roach)?

michaelrog

Document Guzzle options for handling errors & timeouts

8

Whether using symfony or simplehtmldom, if a timeout is set, and the page times outs, it throws and exception and everything stops. is there a way to suppress exceptions?

gixxy22

https://github.com/roach-php/core

michaelrog

Handle Errors (404, 500, etc.)

4

Currently if the scraper gets a 404 or 500 it kills the whole page in a way that isn't recoverable.

codyjames