confluent-kafka-python icon indicating copy to clipboard operation
confluent-kafka-python copied to clipboard

Provide proper documentation and type hints for supported config options

Open palle-k opened this issue 1 year ago • 0 comments

Description

The documentation for the Python library refers to librdkafka's CONFIGURATION.md file when it comes to available config options.

However, this is very far from ideal, as it is not immediately obvious, which C types map to which Python types. Furthermore, this prevents autocompletion of configuration properties.

It would be nice if:

  1. The full configuration is documented for the Python library as well, listing Python types which are required for each property.
  2. The C config is mapped to a Python config TypedDict or Pydantic type, which is then made available through this library. Furthermore, using "." as a separator feels unnatural in Python, where snake case with underscores is preferred.
  3. The config used more precise type annotations, which are then internally mapped to the C config. E.g. bootstrap.servers would possibly be a list[str] instead of a str.

How to reproduce

n.a.

Checklist

Please provide the following information:

  • [x] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): ('2.3.0', 33751040), ('2.3.0', 33751295)
  • [ ] Apache Kafka broker version: n.a.
  • [ ] Client configuration: {...}: n.a.
  • [x] Operating system: macOS 14.1.1 (23B81)
  • [ ] Provide client logs (with 'debug': '..' as necessary): n.a.
  • [ ] Provide broker log excerpts: n.a.
  • [ ] Critical issue

palle-k avatar Nov 29 '23 14:11 palle-k