aiokafka icon indicating copy to clipboard operation
aiokafka copied to clipboard

add typing to aiokafka/record/*

Open dimastbk opened this issue 1 year ago • 3 comments

Changes

Fixes #

Checklist

  • [ ] I think the code is well written
  • [ ] Unit tests for the changes exist
  • [ ] Documentation reflects the changes
  • [ ] Add a new news fragment into the CHANGES folder
    • name it <issue_id>.<type> (e.g. 588.bugfix)
    • if you don't have an issue_id change it to the pr id after creating the PR
    • ensure type is one of the following:
      • .feature: Signifying a new feature.
      • .bugfix: Signifying a bug fix.
      • .doc: Signifying a documentation improvement.
      • .removal: Signifying a deprecation or removal of public API.
      • .misc: A ticket has been closed, but it is not of interest to users.
    • Make sure to use full sentences with correct case and punctuation, for example: Fix issue with non-ascii contents in doctest text files.

dimastbk avatar Apr 21 '24 19:04 dimastbk

Many differences between python and cython implementations.

aiokafka/record/_crecords/legacy_records.pyi:38: error: Class aiokafka.record._crecords.legacy_records.LegacyRecordBatch has abstract attributes "compression_type", "timestamp_type"  [misc]
aiokafka/record/_crecords/legacy_records.pyi:38: note: If it is meant to be abstract, add 'abc.ABCMeta' as an explicit metaclass
aiokafka/record/_crecords/legacy_records.pyi:38: error: Final class aiokafka.record._crecords.legacy_records.LegacyRecordBatch has abstract attributes "compression_type", "timestamp_type"  [misc]
aiokafka/record/_crecords/legacy_records.pyi:42: error: Function is missing a type annotation for one or more arguments  [no-untyped-def]
aiokafka/record/_crecords/legacy_records.pyi:49: error: Class aiokafka.record._crecords.legacy_records.LegacyRecordBatchBuilder has abstract attributes "size_in_bytes"  [misc]
aiokafka/record/_crecords/legacy_records.pyi:49: note: If it is meant to be abstract, add 'abc.ABCMeta' as an explicit metaclass
aiokafka/record/_crecords/legacy_records.pyi:49: error: Final class aiokafka.record._crecords.legacy_records.LegacyRecordBatchBuilder has abstract attributes "size_in_bytes"  [misc]
aiokafka/record/_crecords/default_records.pyi:17: error: Incompatible types in assignment (expression has type "Sequence[Tuple[str, Optional[bytes]]]", base class "DefaultRecordProtocol" defined the type as "List[Tuple[str, Optional[bytes]]]")  [assignment]
aiokafka/record/_crecords/default_records.pyi:30: error: Signature of "timestamp" incompatible with supertype "DefaultRecordProtocol"  [override]
aiokafka/record/_crecords/default_records.pyi:30: note:      Superclass:
aiokafka/record/_crecords/default_records.pyi:30: note:          int
aiokafka/record/_crecords/default_records.pyi:30: note:      Subclass:
aiokafka/record/_crecords/default_records.pyi:30: note:          Optional[int]
aiokafka/record/_crecords/default_records.pyi:32: error: Signature of "timestamp_type" incompatible with supertype "DefaultRecordProtocol"  [override]
aiokafka/record/_crecords/default_records.pyi:32: note:      Superclass:
aiokafka/record/_crecords/default_records.pyi:32: note:          int
aiokafka/record/_crecords/default_records.pyi:32: note:      Subclass:
aiokafka/record/_crecords/default_records.pyi:32: note:          Optional[int]
aiokafka/record/_crecords/default_records.pyi:35: error: Class aiokafka.record._crecords.default_records.DefaultRecordBatch has abstract attributes "attributes", "base_offset", "base_sequence", "crc", "first_timestamp", "last_offset_delta", "magic", "max_timestamp", "next", "producer_epoch", "producer_id", "timestamp_type"  [misc]
aiokafka/record/_crecords/default_records.pyi:35: note: If it is meant to be abstract, add 'abc.ABCMeta' as an explicit metaclass
aiokafka/record/_crecords/default_records.pyi:35: error: Final class aiokafka.record._crecords.default_records.DefaultRecordBatch has abstract attributes "attributes", "base_offset", "base_sequence", "crc", "first_timestamp", "last_offset_delta", "magic", "max_timestamp", "next", "producer_epoch", "producer_id", "timestamp_type"  [misc]
aiokafka/record/_crecords/default_records.pyi:50: error: Class aiokafka.record._crecords.default_records.DefaultRecordBatchBuilder has abstract attributes "write_header"  [misc]
aiokafka/record/_crecords/default_records.pyi:50: note: If it is meant to be abstract, add 'abc.ABCMeta' as an explicit metaclass
aiokafka/record/_crecords/default_records.pyi:50: error: Final class aiokafka.record._crecords.default_records.DefaultRecordBatchBuilder has abstract attributes "write_header"  [misc]
aiokafka/record/_crecords/default_records.pyi:101: error: Class aiokafka.record._crecords.default_records.DefaultRecordMetadata has abstract attributes "crc"  [misc]
aiokafka/record/_crecords/default_records.pyi:101: note: If it is meant to be abstract, add 'abc.ABCMeta' as an explicit metaclass
aiokafka/record/_crecords/default_records.pyi:101: error: Final class aiokafka.record._crecords.default_records.DefaultRecordMetadata has abstract attributes "crc"  [misc]
aiokafka/record/util.py:148: error: Incompatible types in assignment (expression has type "Callable[[Buffer], int]", variable has type "Callable[[Union[array[int], bytes, bytearray, Iterable[int]]], int]")  [assignment]
aiokafka/record/legacy_records.py:474: error: Cannot determine type of "_buffer"  [has-type]
aiokafka/record/legacy_records.py:489: error: Cannot determine type of "_buffer"  [has-type]
aiokafka/record/legacy_records.py:599: error: Can only assign concrete classes to a variable of type "Type[LegacyRecordBatchBuilderProtocol]"  [type-abstract]
aiokafka/record/legacy_records.py:601: error: Can only assign concrete classes to a variable of type "Type[LegacyRecordBatchProtocol]"  [type-abstract]
aiokafka/record/default_records.py:548: error: Unsupported operand types for > ("int" and "None")  [operator]
aiokafka/record/default_records.py:548: note: Left operand is of type "Optional[int]"
aiokafka/record/default_records.py:758: error: Can only assign concrete classes to a variable of type "Type[DefaultRecordBatchBuilderProtocol]"  [type-abstract]
aiokafka/record/default_records.py:759: error: Can only assign concrete classes to a variable of type "Type[DefaultRecordMetadataProtocol]"  [type-abstract]
aiokafka/record/default_records.py:760: error: Can only assign concrete classes to a variable of type "Type[DefaultRecordBatchProtocol]"  [type-abstract]
Found 23 errors in 5 files (checked 35 source files)

dimastbk avatar Apr 21 '24 19:04 dimastbk

https://github.com/aio-libs/aiokafka/blob/master/aiokafka/record/_crecords/default_records.pyx#L452 https://github.com/aio-libs/aiokafka/blob/master/aiokafka/record/_crecords/default_records.pyx#L459

differences from python

dimastbk avatar Apr 24 '24 13:04 dimastbk

Codecov Report

Attention: Patch coverage is 85.47619% with 61 lines in your changes are missing coverage. Please review.

Project coverage is 95.04%. Comparing base (1862620) to head (0c3200b).

Files Patch % Lines
aiokafka/record/_protocols.py 59.71% 0 Missing and 56 partials :warning:
aiokafka/record/default_records.py 96.66% 2 Missing and 1 partial :warning:
aiokafka/record/legacy_records.py 97.40% 2 Missing :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1001      +/-   ##
==========================================
- Coverage   95.36%   95.04%   -0.32%     
==========================================
  Files         112      114       +2     
  Lines       16751    16917     +166     
  Branches     2694     2757      +63     
==========================================
+ Hits        15974    16079     +105     
- Misses        489      493       +4     
- Partials      288      345      +57     
Flag Coverage Δ
cext 91.96% <79.76%> (-0.14%) :arrow_down:
integration 94.69% <85.47%> (-0.32%) :arrow_down:
purepy 94.52% <85.47%> (-0.31%) :arrow_down:
unit 52.78% <85.47%> (+0.11%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Apr 25 '24 13:04 codecov[bot]