brod icon indicating copy to clipboard operation
brod copied to clipboard

:brod.create_topics topic config?

Open jamespeterschinner opened this issue 4 years ago • 5 comments
trafficstars

Hey folks,

I've have a use case for the create_topics and delete_topics in my test cases.

https://github.com/kafka4beam/brod/pull/348

I'm using brod 3.16 in an Elixir context, and running the above example in the shell gives me an error:

topic_config = [
  %{
    config_entries: [],
    num_partitions: 2,
    replica_assignment: [],
    replication_factor: 1,
    topic: "test_create_topic"
  }
]
:brod.create_topics([{'localhost', 9092}], topic_config, %{})
:brod.delete_topics([{'localhost', 9092}], ["test_create_topic"], 1_000)
iex(10)> :brod.create_topics([{'localhost', 9092}], topic_config, %{})
** (ErlangError) Erlang error: {:field_missing, [stack: [{:create_topics, 0}, :topics, :name], input: %{config_entries: [], num_partitions: 2, replica_assignment: [], replication_factor: 1, topic: "test_create_topic"}]}

Thanks for the great work in creating brod 👍

jamespeterschinner avatar Aug 23 '21 05:08 jamespeterschinner

Hey!

Based in the errors, I managed to get it working with:

[
  %{
    assignments: [],
    configs: [],
    name: "sample",
    num_partitions: 1,
    replication_factor: 1
  }
]

Here are the schemas for this request: https://github.com/kafka4beam/kafka_protocol/blob/9ee00f2efa97df23c13d12a48196bed11e183f41/src/kpro_schema.erl#L582

Which makes me think there is some documentation error, as any of the schemas use the documented names.

Perhaps someone from the team knows better? If it is misdocumented, I can send a PR

v0idpwn avatar Aug 23 '21 22:08 v0idpwn

Hi.

topic_config = [
  %{
    config_entries: [],
    num_partitions: 2,
    replica_assignment: [],
    replication_factor: 1,
    topic: "test_create_topic"
  }
]

this seem to be the old layout which should work with brod 3.15 brod 3.16 has kafka_protocol upgraded to 4.0, there were quite some non-backward compatible schema changes.

I'm not sure if this implicit API contract is documented anywhere.

zmstone avatar Aug 23 '21 22:08 zmstone

Thanks for the quick reply folks!

jamespeterschinner avatar Aug 24 '21 00:08 jamespeterschinner

I'm a bit confused here: should we update these readme example? If yes, I can submit a PR.

v0idpwn avatar Sep 03 '21 19:09 v0idpwn

yes. we should update the readme

zmstone avatar Sep 14 '21 19:09 zmstone