pykafka
pykafka copied to clipboard
[WIP] Support RecordBatch
This pull request addresses https://github.com/Parsely/pykafka/issues/831 by adding support for RecordBatch message formatting to pykafka. This includes supporting the encoding and decoding of varints via the struct_helpers interface.
Codecov Report
:exclamation: No coverage uploaded for pull request base (
master@8747398). Click here to learn what that means. The diff coverage is75.71%.
@@ Coverage Diff @@
## master #844 +/- ##
=========================================
Coverage ? 82.38%
=========================================
Files ? 36
Lines ? 3850
Branches ? 529
=========================================
Hits ? 3172
Misses ? 526
Partials ? 152
| Impacted Files | Coverage Δ | |
|---|---|---|
| pykafka/protocol/message.py | 79.87% <21.05%> (ø) |
|
| pykafka/utils/struct_helpers.py | 97.95% <96.07%> (ø) |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update 8747398...7370c9d. Read the comment docs.
The protocol logic for RecordBatch is implemented and untested. The next step is to figure out how to get Kafka to start speaking in RecordBatch instead of MessageSet so I can run some tests to work out the protocol implementation bugs. Once that's done, the next step will be to make changes to the producer and consumer to have them use RecordBatch when appropriate.