Several handlers

[rom8726]

Hi! I have several different independent Kafka clusters. Is it possible to configure several message handlers for each cluster independently in one application?

[objectuser]

@rom8726 Kaffe doesn't currently support a single consumer with different connection settings. I think we have an older issue related to separating out those settings.

The only way I can think of to do that currently is to use an umbrella app.

[madshargreave]

@objectuser is it possible to have multiple consumers using the same connection settings? I tried to umbrella-app approach, but it does not work as the Kaffe configuration is overridden across sub-apps

[objectuser]

@madshargreave That's unfortunate that the umbrella approach doesn't work.

I think multiple consumers would have the same problem.

We're interested in supporting multiple consumers but just haven't had time. Would you say #14 is essentially what you want?

[madshargreave]

@objectuser Yes , being able to specify a message processor for each topic would be perfect

[objectuser]

👍

I recently refactored a project into an umbrella. I needed a separate configuration (not for Kaffe) for one of the projects. I was surprised how much umbrella projects want to use the same config everywhere.

What I did in this case was to point mix.exs to a "local" configuration:

  def project do
    [
      config_path: "config/local.exs",
      deps_path: "../../deps",
      lockfile: "../../mix.lock",
      elixir: "~> 1.6",
      ...
    ]
  end

That file, config/local.exs, had project-specific configuration items, but then defered to the global config:

use Mix.Config

config :special_config,
  so_special: :do_that_thing

import_config "../../../config/config.exs"

I've not tried this, but I wonder if the Kaffe configuration could replace the :special_config in each sub-project, allowing a separate consumer config in each.

[madshargreave]

I see, I think I could do something like this for now, but it would be awesome if Kaffe managed it internally in the future

Thanks!