hermann

hermann copied to clipboard

Instead of fprintf, C extension should use a Ruby logger

1

Right now `hermann_lib.c` has a number of calls to `fprintf` to record information/logging info to `stderr`. It really should send that information to a Ruby logger.

rtyler

It is useful to check for producer.connected? or producer.errored? before publishing messages. Otherwise we may want to create a fresh instance of producer. ``` def get_producer if @producer.nil? || @producer.errored?...

rtyler

Update producer default settings

1

``` ruby #Current defaults DEFAULTS = { 'partitioner.class' => 'kafka.producer.DefaultPartitioner', 'request.required.acks' => '1', 'message.send.max.retries' => '0' }.freeze ``` I'm thinking we should make: `request.required.acks` = -1 (needs ack from all...

jamescway

Effectively implementing te JRuby corrolary to #95

rtyler

Option to consume one message at a time? And/or configurable timeout to exit consumption?

1

This question is targeted to the MRI version, although it can be asked of JRuby as well. Is it currently possible? From a brief look at the sample code in...

daluu

I had a misconfiguration with my Hermann consumer which caused an exception to occur: ``` Hermann::Provider::JavaSimpleConsumer#consume EXCEPTION java.lang.IllegalArgumentException: Invalid path string "/brokers/topics//auditlog_events" caused by empty node name specified @16 ```...

blaines

Useful for standing up an environment from scratch. In JRuby I can hack it with something like Java::Kafka::admin::TopicCommand.main(["--zookeeper", $zookeepers, "--create", "--topic", $topic, "--partitions", "1", "--replication-factor", "1"].to_java(:string)) But there's no workaround...

lsb

`JavaSimpleConsumer#create_config` should really be receiving the options passed to the initializer. DERP.

rtyler

Producer#push with an array of messages should call rd_kafka_produce_batch

1

Instead of calling `rd_kafka_produce` through `Herman::Lib::Producer#push_single` in a tight loop, we should call out to `rd_kafka_produce_batch` which is a better suited batch API.

rtyler

Investigate using FFI instead of writing C extensions

1

[FFI](https://github.com/ffi/ffi) looks like it has matured enough to where we could delete all the custom C code in this repository and just use FFI bindings. Couple of open questions: 1....

rtyler