Ewen Cheslack-Postava

Yeah, I think arbitrary combos is unlikely to be useful in practice. If you're trying to use JSON keys and Avro values, you have bigger problems. Once you start with...

@criccomini Query param doesn't seem awful. But making a bunch of them for different types doesn't seem ideal. (And of course that doesn't really affect the fact that we need...

@johnpaullarsen Great point. We designed around the index-per-day pattern mentioned at the beginning of that post because we know it's very widely used, but this approach should be supported as...

@johnpaullarsen Yes, you can expect an update around end of Feb.

Yeah, the challenge with these pipelines is that it can sometimes be difficult to find the real source of the problem. It's possible this will be fixed in the next...

@dalejin2014 We'd love to have native stream processing libraries in different languages and having really good Kafka clients is the basis for that. That said, we don't have a timeline...

@xiabai84 Your problem seems unrelated. It looks like you have data in the Kafka Connect config topic that is in a different format than expected. This could happen if you...

@heifrank Note that `flush.size` is *per topic partition* not *per topic*. This can be important if you had 16k messages in your topic, but more than 16 partitions since you...

You're not doing it wrong, you're just running into some simplifying assumptions that we haven't had to update for almost 4 years now :) We started out with a super...

For 1, you should just be able to use the `cluster_spec` that's already passed into `Service`, see https://github.com/confluentinc/ducktape/blob/master/ducktape/services/service.py#L54 and how we resolve between `num_nodes` vs `cluster_spec`: https://github.com/confluentinc/ducktape/blob/master/ducktape/services/service.py#L97. The `num_nodes` is...