CommonCrawler icon indicating copy to clipboard operation
CommonCrawler copied to clipboard

More detailed logging upon failure(s)

Open ChrisCates opened this issue 7 years ago • 2 comments
trafficstars

Sometimes the network can fail or other things can happen... However, we don't have detailed logs for when a failure happens for a specific wet file... Would be nice to have detail logs in the output:

  • [x] Better logging for extract.go

  • [ ] Better logging for analyze.go

ChrisCates avatar Oct 10 '18 04:10 ChrisCates

For more detailed logging I suggest to:

  1. replace CURL and gunzip calls with go code;
  2. return error values instead of books;
  3. move logging to higher level functions.

I made a pr #3 illustrating this ideas in code

LastPossum avatar Oct 11 '18 15:10 LastPossum

@LastPossum thank you for the pull request. It has been merged... I will look into using more idiomatic error outputs when I do another push.

ChrisCates avatar Oct 11 '18 19:10 ChrisCates