Fix: Removed indentation from statistics logging
Description
The purpose of the PR is to fix the indentation of statistics logging. It was originally 8 space indentation but now it is changed to be all on one line.
Issues
This fixes issue #306
Testing
I went through the unit test files and logged out what the output would be for the statistics loggin.
Example:
[crawlee.basic_crawler.basic_crawler] INFO Final request statistics: {"requests_finished": 3, "requests_failed": 0, "retry_histogram": [3], "request_avg_failed_duration": null, "request_avg_finished_duration": 0.000984, "requests_finished_per_minute": 7348, "requests_failed_per_minute": 0, "request_total_duration": 0.002951, "requests_total": 3, "crawler_runtime": 0.024498}
Checklist
- [ ] Changes are described in the
CHANGELOG.md - [x] CI passed
FYI we had a quick poll about this and in the end we agreed we want to print this as a formatted table instead of JSON, so we won't be merging this one.
Something like the following (but with snake_case instead of camelCase - this snippet was built based on the JS version)
┌──────────────────────────────────┬────────┐
│ requestsFinished │ 59 │
│ requestsFailed │ 0 │
│ retryHistogram │ [ 59 ] │
│ requestAvgFailedDurationMillis │ null │
│ requestAvgFinishedDurationMillis │ 1071 │
│ requestsFinishedPerMinute │ 67 │
│ requestsFailedPerMinute │ 0 │
│ requestTotalDurationMillis │ 63215 │
│ requestsTotal │ 59 │
│ crawlerRuntimeMillis │ 53197 │
└──────────────────────────────────┴────────┘
We should be able to print tables using rich.Table - see https://rich.readthedocs.io/en/stable/tables.html
Thanks for your feedback! I am brand new to open source. I did make some furthers changes by using a table to print it out as a formatted table. Should I open a new PR or update this one?
Thanks for your feedback! I am brand new to open source. I did make some furthers changes by using a table to print it out as a formatted table. Should I open a new PR or update this one?
Awesome! Just commit the changes you made and push it to your fork and the PR will be updated automatically