Jon Slusher

Results 22 comments of Jon Slusher

Thanks for looking into the issue! I'm a little confused about how to proceed now that this issue is closed. Is there a patch version for `kombu` on the way?...

In my case, the only way I've found I can make these warnings go away is to reduce `batch.size` to a point where they stop. I'm under the impression that...

@martin-g The python module I'm referring to is the [confluent-kafka-python](https://github.com/confluentinc/confluent-kafka-python) module. Versions 1.9.2 and later respect Avro's [permissive setting](https://issues.apache.org/jira/browse/NIFI-4612). Take a look [at the link](https://issues.apache.org/jira/browse/NIFI-4612) and you'll see what I...

> Since than the link you shared, has nothing to do with the actual issue? I shared a couple of links. To which are you referring? If it's the [Apache...

@martin-g I had initially confused that link for the issue that made Avro itself permissive, but I see now that it's the NIFI project allowing a permissive setting. Apologies for...

Here is the error we're getting: ``` thread 'main' panicked at /app/src/utils/kafka.rs:67:35: Error decoding value: Error: Supplied raw value "{\"type\":\"record\",\"name\":\"Envelope\",\"namespace\":\"debezium.abc-123-efg-20231005.table.u_table_dbz\",\"fields\":[{\"name\":\"before\",\"type\":[\"null\",{\"type\":\"record\",\"name\":\"Value\",\"fields\":[{\"name\":\"id\",\"type\":\"int\"},{\"name\":\"uid\",\"type\":{\"type\":\"long\",\"connect.default\":0},\"default\":0},{\"name\":\"release_id\",\"type\":{\"type\":\"long\",\"connect.default\":0},\"default\":0},{\"name\":\"notes\",\"type\":\"string\"},{\"name\":\"notes_public\",\"type\":{\"type\":\"string\",\"connect.version\":1,\"connect.parameters\":{\"allowed\":\"Y,N\"},\"connect.default\":\"N\",\"connect.name\":\"io.debezium.data.Enum\"},\"default\":\"N\"},{\"name\":\"added_ts\",\"type\":{\"type\":\"long\",\"connect.version\":1,\"connect.name\":\"io.debezium.time.Timestamp\"}}],\"connect.name\":\"debezium.abc-123-efg-20231005.table.u_table_dbz.Value\"}],\"default\":null},{\"name\":\"after\",\"type\":[\"null\",\"Value\"],\"default\":null},{\"name\":\"source\",\"type\":{\"type\":\"record\",\"name\":\"Source\",\"namespace\":\"io.debezium.connector.mysql\",\"fields\":[{\"name\":\"version\",\"type\":\"string\"},{\"name\":\"connector\",\"type\":\"string\"},{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"ts_ms\",\"type\":\"long\"},{\"name\":\"snapshot\",\"type\":[{\"type\":\"string\",\"connect.version\":1,\"connect.parameters\":{\"allowed\":\"true,last,false,incremental\"},\"connect.default\":\"false\",\"connect.name\":\"io.debezium.data.Enum\"},\"null\"],\"default\":\"false\"},{\"name\":\"db\",\"type\":\"string\"},{\"name\":\"sequence\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"table\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"server_id\",\"type\":\"long\"},{\"name\":\"gtid\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"file\",\"type\":\"string\"},{\"name\":\"pos\",\"type\":\"long\"},{\"name\":\"row\",\"type\":\"int\"},{\"name\":\"thread\",\"type\":[\"null\",\"long\"],\"default\":null},{\"name\":\"query\",\"type\":[\"null\",\"string\"],\"default\":null}],\"connect.name\":\"io.debezium.connector.mysql.Source\"}},{\"name\":\"op\",\"type\":\"string\"},{\"name\":\"ts_ms\",\"type\":[\"null\",\"long\"],\"default\":null},{\"name\":\"transaction\",\"type\":[\"null\",{\"type\":\"record\",\"name\":\"block\",\"namespace\":\"event\",\"fields\":[{\"name\":\"id\",\"type\":\"string\"},{\"name\":\"total_order\",\"type\":\"long\"},{\"name\":\"data_collection_order\",\"type\":\"long\"}],\"connect.version\":1,\"connect.name\":\"event.block\"}],\"default\":null}],\"connect.version\":1,\"connect.name\":\"debezium.abc-123-efg-20231005.table.u_table_dbz.Envelope\"}" cant be turned into a Schema, was cause by Invalid...

@gklijs Apologies, but I actually shared the wrong schema earlier. The schema that this consumer is trying to use does have a namespace with hyphens: ``` { "type": "record", "name":...

@martin-g We use [confluent_python[avro]](https://github.com/confluentinc/confluent-kafka-python). Versions 1.9.2 or greater seem to support those namespaces. The [example in their repo](https://github.com/confluentinc/confluent-kafka-python/blob/master/examples/avro_consumer.py), using `AvroDeserializer` is close enough to what we have.

I appreciate your working through this with me. You're saying that rust's Avro SDK is what isn't supporting these names? I see in the [serializer section of the Python module](https://github.com/confluentinc/confluent-kafka-python/blob/master/src/confluent_kafka/avro/serializer/message_serializer.py#L29-L30)...

@martin-g Thank you for hearing me out! I opened [this issue (AVRO-3900)](https://issues.apache.org/jira/browse/AVRO-3900) with the request. Let me know if you need any more information there to proceed.