onetl
onetl copied to clipboard
[DOP-18743] Set default jobDescription
Change Summary
Set jobDescription in any method which interact with Spark in some way. Examples:
This does not work if Spark action was triggered outside of onETL methods, e.g. user called df.count() or df.toPandas():
It's hard to write tests on this because jobDescription is reset just before method returned some result.
Related issue number
Checklist
- [X] Commit message and PR title is comprehensive
- [X] Keep the change as small as possible
- [ ] Unit and integration tests for the changes exist
- [X] Tests pass on CI and coverage does not decrease
- [ ] Documentation reflects the changes where applicable
- [X]
docs/changelog/next_release/<pull request or issue id>.<change type>.rstfile added describing change (see CONTRIBUTING.rst for details.) - [x] My PR is ready to review.
Codecov Report
Attention: Patch coverage is 97.53086% with 4 lines in your changes missing coverage. Please review.
Project coverage is 95.45%. Comparing base (
3c25405) to head (e34d9e5). Report is 57 commits behind head on develop.
Additional details and impacted files
@@ Coverage Diff @@
## develop #304 +/- ##
===========================================
+ Coverage 95.36% 95.45% +0.09%
===========================================
Files 225 225
Lines 8860 8931 +71
Branches 1499 1516 +17
===========================================
+ Hits 8449 8525 +76
+ Misses 292 287 -5
Partials 119 119
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.