hadoop-connectors
hadoop-connectors copied to clipboard
Handle partial write update from GCS
Connector supports resuming upload only from MAX_BYTES_PER_MESSAGE boundary. If GCS reports a committed offset which is not a multiple of this number, then we cannot resume.
/gcbrun
Codecov Report
Merging #781 (71ecf70) into master (d465885) will decrease coverage by
18.33%
. The diff coverage is0.00%
.
@@ Coverage Diff @@
## master #781 +/- ##
=============================================
- Coverage 81.23% 62.89% -18.34%
+ Complexity 1978 1446 -532
=============================================
Files 133 133
Lines 8818 8822 +4
Branches 1025 1026 +1
=============================================
- Hits 7163 5549 -1614
- Misses 1240 2735 +1495
- Partials 415 538 +123
Flag | Coverage Δ | |
---|---|---|
integrationtest | 62.89% <0.00%> (-0.15%) |
:arrow_down: |
unittest | ? |
Flags with carried forward coverage won't be shown. Click here to find out more.
Impacted Files | Coverage Δ | |
---|---|---|
...doop/gcsio/GoogleCloudStorageGrpcWriteChannel.java | 64.46% <0.00%> (-15.09%) |
:arrow_down: |
...a/com/google/cloud/hadoop/util/AccessBoundary.java | 0.00% <0.00%> (-100.00%) |
:arrow_down: |
...om/google/cloud/hadoop/util/IoExceptionHelper.java | 0.00% <0.00%> (-100.00%) |
:arrow_down: |
...le/cloud/hadoop/io/bigquery/ShardedInputSplit.java | 0.00% <0.00%> (-100.00%) |
:arrow_down: |
...d/hadoop/gcsio/testing/GcsItemInfoTestBuilder.java | 0.00% <0.00%> (-100.00%) |
:arrow_down: |
...bigquery/output/FederatedBigQueryOutputFormat.java | 0.00% <0.00%> (-100.00%) |
:arrow_down: |
...query/output/FederatedBigQueryOutputCommitter.java | 0.00% <0.00%> (-100.00%) |
:arrow_down: |
...gcsio/StorageRequestToAccessBoundaryConverter.java | 0.00% <0.00%> (-98.77%) |
:arrow_down: |
...d/hadoop/util/testing/MockHttpTransportHelper.java | 0.00% <0.00%> (-96.67%) |
:arrow_down: |
...adoop/io/bigquery/DynamicFileListRecordReader.java | 0.00% <0.00%> (-93.60%) |
:arrow_down: |
... and 70 more |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update d465885...71ecf70. Read the comment docs.