gobblin
gobblin copied to clipboard
GOBBLIN-1055 Gaas jobs not able to write to state store
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA
- [X] My PR addresses the following [Gobblin JIRA]
- https://issues.apache.org/jira/browse/GOBBLIN-1055
Description
- [X] Here are some details about my PR, including screenshots (if applicable):
The change pass the GobblinClassLoader to Hadoop through Configuration so that Hadoop can load the serializer properly.
Tests
- [X] My PR adds the following unit tests OR does not need testing for this extremely good reason:
PR fixed a loop hole in passing parameters from Gobblin-metastore to Hadoop, and it was tested effective in Gaas flow execution with state.store.enabled=true.
Commits
- [X] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
- Subject is separated from body by a blank line
- Subject is limited to 50 characters
- Subject does not end with a period
- Subject uses the imperative mood ("add", not "adding")
- Body wraps at 72 characters
- Body explains "what" and "why", not "how"
Codecov Report
Merging #2895 into master will increase coverage by
<.01%. The diff coverage is100%.
@@ Coverage Diff @@
## master #2895 +/- ##
============================================
+ Coverage 45.82% 45.82% +<.01%
+ Complexity 9151 9149 -2
============================================
Files 1929 1929
Lines 72625 72626 +1
Branches 7995 7995
============================================
+ Hits 33279 33281 +2
+ Misses 36309 36308 -1
Partials 3037 3037
| Impacted Files | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| ...ava/org/apache/gobblin/metastore/FsStateStore.java | 36.48% <100%> (+0.43%) |
14 <0> (ø) |
:arrow_down: |
| ...e/gobblin/runtime/locks/ZookeeperBasedJobLock.java | 63.33% <0%> (-1.12%) |
15% <0%> (-1%) |
|
| .../org/apache/gobblin/cluster/GobblinTaskRunner.java | 64.62% <0%> (-0.44%) |
28% <0%> (-1%) |
|
| ...lin/restli/throttling/ZookeeperLeaderElection.java | 72.22% <0%> (+2.22%) |
13% <0%> (ø) |
:arrow_down: |
| ...lin/util/filesystem/FileSystemInstrumentation.java | 92.85% <0%> (+7.14%) |
3% <0%> (ø) |
:arrow_down: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update 2991f42...540c734. Read the comment docs.