Add ability to persist logs into a file instead of logging them

Open atomicptr opened this issue 5 years ago • 0 comments

$ crab crawl:sitemap https://domain.com/sitemap.xml --output output.log
# NO prints here...

It might probably make sense to have a different option for persisting the logs as a valid json file...

$ crab crawl:sitemap https://domain.com/sitemap.xml --output-json output.json
$ cat output.json
[
    {"status": 200, "url": "https://domain.com/page-1", ...},
    ...
]