data-prep-kit
data-prep-kit copied to clipboard
[Logging Feature] Logging INFO about completed x files in y mins should add (xx1 successfully and xx2 failed)
Search before asking
- [X] I searched the issues and found no similar issues.
Component
Library/core
Feature
When running a transform, the logging INFO on my terminal reports something like this:
10:25:50 WARNING - Failed extraction. Exception 400 Client Error: Bad Request for url: http://main-service-multi.apps.fmaas-instructlab-01.fmaas.res.ibm.com:80/api/v1/task/process processing file empty.pdf, skipping
10:25:50 INFO - ['empty.pdf', 'empty.pdf']=
10:25:50 INFO - Completed 1 files (25.0%) in 0.007229816913604736 min
Although this is semantically arguable that it is correct in the sense that 1 file is completed, even though that file is not successfully completed. It would be nice to add additional logging info to expand about how many are successful and how many failed. For example, it would be nice to say something like:
10:25:50 INFO - Completed 1 files (25.0%; 0 are successful; 1 failed) in 0.007229816913604736 min
Additionally, some of the time and % are printed out with too many digits. I would be nice to limit to about 2 decimal digits. For example:
10:29:04 INFO - Completed execution in 3.2514644304911298 min, execution result 0
would be better to print out like:
10:29:04 INFO - Completed execution in 3 min 15.08 sec, execution result 0
.
Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!