onetl icon indicating copy to clipboard operation
onetl copied to clipboard

[DOP-18743] Set default jobDescription

Open dolfinus opened this issue 1 year ago • 1 comments

Change Summary

Set jobDescription in any method which interact with Spark in some way. Examples: Снимок экрана_20240808_192304 Снимок экрана_20240808_192326

This does not work if Spark action was triggered outside of onETL methods, e.g. user called df.count() or df.toPandas(): Снимок экрана_20240808_192425

It's hard to write tests on this because jobDescription is reset just before method returned some result.

Related issue number

Checklist

  • [X] Commit message and PR title is comprehensive
  • [X] Keep the change as small as possible
  • [ ] Unit and integration tests for the changes exist
  • [X] Tests pass on CI and coverage does not decrease
  • [ ] Documentation reflects the changes where applicable
  • [X] docs/changelog/next_release/<pull request or issue id>.<change type>.rst file added describing change (see CONTRIBUTING.rst for details.)
  • [x] My PR is ready to review.

dolfinus avatar Aug 08 '24 16:08 dolfinus

Codecov Report

Attention: Patch coverage is 97.53086% with 4 lines in your changes missing coverage. Please review.

Project coverage is 95.45%. Comparing base (3c25405) to head (e34d9e5). Report is 57 commits behind head on develop.

Files with missing lines Patch % Lines
...etl/connection/db_connection/mongodb/connection.py 77.77% 1 Missing and 1 partial :warning:
onetl/connection/file_connection/ftp.py 66.66% 1 Missing :warning:
onetl/file/file_df_reader/file_df_reader.py 91.66% 0 Missing and 1 partial :warning:
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #304      +/-   ##
===========================================
+ Coverage    95.36%   95.45%   +0.09%     
===========================================
  Files          225      225              
  Lines         8860     8931      +71     
  Branches      1499     1516      +17     
===========================================
+ Hits          8449     8525      +76     
+ Misses         292      287       -5     
  Partials       119      119              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Aug 08 '24 17:08 codecov[bot]