confluent-kafka-python icon indicating copy to clipboard operation
confluent-kafka-python copied to clipboard

How to mock SerializingProducer?

Open liveFreeOrCode opened this issue 3 years ago • 1 comments

Description

confluent_kafka versions: image

In bootstrapping our application, we create a SerializingProducer and dump it into our applications registry (registry not relevant)

schema_registry_client = SchemaRegistryClient({
    "url": "https://fake.cc.schema.registry.com",
    "basic.auth.user.info": "API_KEY:API_SECRET",
})

pb_serializer = ProtobufSerializer(
    ExampleRecord,
    schema_registry_client,
    {"use.deprecated.format": True},
)

producer = SerializingProducer({
    "bootstrap.servers": "https://fake.cc.schema.registry.com",
    "key.serializer": StringSerializer("utf_8"),
    "value.serializer": pb_serializer,
    "security.protocol": "SASL_SSL",
    "sasl.mechanisms": "PLAIN",
    "sasl.username": "username",
    "sasl.password": "password",
})

"""Add a Kafka producer clients to the registry."""
registry["kafka_example_producer"] = producer

In testing, with the use of pytest and (pytest-mock just for the mocker fixture) we do this:

@pytest.fixture(autouse=True)
def mock_kafka_producer(mocker):
    return mocker.patch("confluent_kafka.SerializingProducer")

However, when running our tests, we are still getting:

Screen Shot 2022-05-17 at 12 53 21 PM

My question is, is this the proper way to mock SerializingProducer, or am I way off. I wasn't successful in finding any examples in this libraries ./tests folder.

How to reproduce

Checklist

Please provide the following information:

  • [x] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()):
  • [ ] Apache Kafka broker version:
  • [x] Client configuration: {...}
  • [ ] Operating system:
  • [x] Provide client logs (with 'debug': '..' as necessary)
  • [ ] Provide broker log excerpts
  • [ ] Critical issue

liveFreeOrCode avatar May 17 '22 17:05 liveFreeOrCode

I'm not sure. Marking as enhancement as testability is something we should think through properly before marking this api as stable. SerializingProducer derives from the librdkafka based producer, so constructing it is going to make an actual producer that tries to connect to kafka etc.

mhowlett avatar May 18 '22 14:05 mhowlett

We now have stabilized everything in the serde API except SerializingProducer / DeserializingConsumer, which is still marked as "experimental". We anticipate deprecating these classes in the future. Refer to the examples for how to use the serdes independent of the SerializingProducer.

mhowlett avatar Oct 25 '22 13:10 mhowlett