Add a max pages option

[bejean]

Add a max pages number option. Should this be the maximum number of pages fetched on the server or the max number of pages sent to the pipeline ? This can be very different. https://groups.google.com/forum/#!topic/crawl-anywhere/Rcb5zricvTo

[aravinuthala]

It has to be the max number of documents sent to the pipeline, as a fetch from the server need not be written to the pipeline.