Dan Nguyen

I think the better solution is to be agnostic about whether the page is an "index" or a data-page and let the user define via blocks how the scraper should...

@kgrz I think it's possible to keep the concept of #index and #instance type pages in the public facing API, as conveniences. But under the hood, there should be no...

In any case, I think rewriting and reorganizing the tests, as per #6 , should be the proper course of action before radically refactoring the API. If it seems that...

@jeremybmerrill I'd be careful with creating a chaining concept without getting a better idea of how people approach scraping beyond the simple scenario of indexpage -> data page. In the...

Come to think of it, though, how useful is it for the user to recognize the stashed filenames? If the scrape goes well, they shouldn't ever have to worry about...

Yeah, one thing that just came to mind...what has been useful to me is to have, if not _readable_ cache-file names, **reversible** cache names...(the solution just ends up being both)....

It's not the test framework that is slow, it's the use of a webserver. At some point, the dependence on Thin should be removed and replaced with Fakeweb to simulate...

Yeah...including the full pages makes things slower on two fronts...one, with just the opening and parsing of the pages, and two, for other contributors to read. For example, some of...