confluent-kafka-python
confluent-kafka-python copied to clipboard
confluentinc
Confluent's Kafka Python Client
This is a question, not an issue. We've been using the normal rebalancing strategy, whereby n consumer threads are launched and allocated to partitions by kafka. However, this has occasionally...
Description =========== I have a topic with 24 partitions which is produced about 1000 messages/sec. The cosumer-commit-offset is manual. 1.After starting consumer, the Broker logs : "Preparing to rebalance group·····...
Description =========== When I set "group.initial.rebalance.delay.ms=10000" on broker version "3.1.0", the messages polled by method `consumer.poll(timeout=1.0)` sometimes with from very high number offset() (such as 4272995135), even though the maximum...
Hello 👋 I'm responsible for two App Containers consuming from 2 different Kafka topics: one with Avro schema, the other schema-less. The schema less consumer never crashed, however the one...
Description =========== I have a problem with a producer in an API. Specifically, once in a while it'll throw errors like `SASL authentication error: SaslAuthenticateRequest failed: Local: Broker transport failure...
Description =========== SchemaRegistryClient does not supply a timeout to the underlying calls to `requests`. If a network issue causes no data to be returned (such as firewall/security group misconfiguration), the...
Description =========== Support for multiple schemas in one topic for JAVA avro serializer was added in pull request: [https://github.com/confluentinc/schema-registry/pull/680](url) Any chance this could be supported from python too? How to...
Fixes https://github.com/confluentinc/confluent-kafka-python/issues/974 To use Serializer: ``` named_schemas = {} ref_dict = loads(ref_schema_str) parse_schema(ref_dict, _named_schemas=named_schemas) ref_schema = SchemaReference("org.test.MyReferencedItem", "referenced-subject", "1") schema = Schema(schema_str, "AVRO", [ref_schema]) avro_serializer = AvroSerializer(schema_registry_client, schema, None, None,...
## Overview When using the Cached Schema Registry client with a URL that has moved or been decommissioned, calls to AvroConsumer.poll() will raise a SerializerError. This is the same error...
Description =========== Here the `timeout` option is hardcoded as `-1` and not surfaced to the Python API. https://github.com/confluentinc/confluent-kafka-python/blob/e671bccb8a4f98302748ccf60d5d579f68c6613d/src/confluent_kafka/src/Consumer.c#L817 Would be great to be able to set `timeout` in this API....
