crawl-anywhere
crawl-anywhere copied to clipboard
Post processing API
Just checked the Web search php application and it seems like an ideal base for a post processing API that could have calls like
Get URL contents Get ALL domains etc.