snowflake-ingest-java icon indicating copy to clipboard operation
snowflake-ingest-java copied to clipboard

@snow SNOW-657667 Streaming ingest: factor out channel from the buffer

Open sfc-gh-azagrebin opened this issue 2 years ago • 6 comments

SNOW-657667

clean up refactoring of the circular dependencies: channel -> buffer -> channel channel -> chunk data -> channel this allows to use buffer independently from channel and client, e.g. in tests and test file generator

sfc-gh-azagrebin avatar Nov 10 '22 19:11 sfc-gh-azagrebin

@sfc-gh-gdoci Thanks for the review Gloria! Addressed!

sfc-gh-azagrebin avatar Nov 11 '22 14:11 sfc-gh-azagrebin

Any idea why we don't have the code coverage report for this PR?

sfc-gh-tzhang avatar Nov 15 '22 00:11 sfc-gh-tzhang

Any idea why we don't have the code coverage report for this PR?

No idea, this is something new for me that I saw only couple of times last week. Could it be some timing issue with when the PR was open? Do you know who is introduced the code coverage report?

sfc-gh-azagrebin avatar Nov 15 '22 11:11 sfc-gh-azagrebin

Codecov Report

Merging #285 (0c706ad) into master (c7cc0a8) will increase coverage by 0.02%. The diff coverage is 90.71%.

@@            Coverage Diff             @@
##           master     #285      +/-   ##
==========================================
+ Coverage   76.61%   76.63%   +0.02%     
==========================================
  Files          71       73       +2     
  Lines        4867     4914      +47     
  Branches      454      455       +1     
==========================================
+ Hits         3729     3766      +37     
- Misses        918      928      +10     
  Partials      220      220              
Impacted Files Coverage Δ
...lake/ingest/streaming/internal/ArrowRowBuffer.java 93.84% <50.00%> (-0.04%) :arrow_down:
...ingest/streaming/internal/ChannelFlushContext.java 67.74% <67.74%> (ø)
...owflake/ingest/streaming/internal/ChannelData.java 97.50% <83.33%> (+0.20%) :arrow_up:
...e/ingest/streaming/internal/AbstractRowBuffer.java 93.19% <88.23%> (-0.87%) :arrow_down:
...owflake/ingest/streaming/internal/BlobBuilder.java 91.20% <96.15%> (+6.59%) :arrow_up:
...wflake/ingest/streaming/internal/ArrowFlusher.java 98.00% <100.00%> (+0.04%) :arrow_up:
...ake/ingest/streaming/internal/ChannelMetadata.java 100.00% <100.00%> (ø)
...ingest/streaming/internal/ChannelRuntimeState.java 100.00% <100.00%> (ø)
...flake/ingest/streaming/internal/ChunkMetadata.java 100.00% <100.00%> (ø)
...wflake/ingest/streaming/internal/FlushService.java 74.04% <100.00%> (-3.62%) :arrow_down:
... and 5 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov-commenter avatar Nov 15 '22 12:11 codecov-commenter

Suddenly there is a coverage report today.

sfc-gh-azagrebin avatar Nov 15 '22 15:11 sfc-gh-azagrebin

Thanks Toby! I responded to comments

sfc-gh-azagrebin avatar Nov 16 '22 12:11 sfc-gh-azagrebin

These changes are great, I prefer to split the functionality into more classes, immutable when possible. The ability to construct a buffer without a corresponding channel is really nice. Thanks a lot for for the work, @sfc-gh-azagrebin!

sfc-gh-lsembera avatar Nov 30 '22 13:11 sfc-gh-lsembera