azure-functions-kafka-extension icon indicating copy to clipboard operation
azure-functions-kafka-extension copied to clipboard

Can you include a sample for dotnet-isolated, which works for both localhost via docker and also eventHubs?

Open justinmchase opened this issue 3 years ago • 2 comments

In the current samples: https://github.com/Azure/azure-functions-kafka-extension/tree/dev/samples/dotnet-isolated

You have a lot of settings hard coded right into the various trigger attributes.

But I need to support multiple environments, local and running in azure where I have to use environment variables to change the configuration. Its not clear here how the hard coded attribute settings are supposed to be resolving environment variables for the configs... I would be hugely helpful to have a real world sample for "local" where you can simply change what its connected to purely by changing the appsettings file.

justinmchase avatar May 10 '22 17:05 justinmchase

@justinmchase, please view the localsettings.json.example and map that with the kafkatrigger annotation of dotnet isolated you will be able to figure out your solution. The localsettings.json entries can be environment variables, you don't require that also.

Regarding localbroker we don't recommend that in Production but if you still wants to try please check this link -- https://github.com/Azure/azure-functions-kafka-extension/blob/f154788fcd7e10de5bfe5b7f738a7d35b98fa469/samples/dotnet/KafkaFunctionSample/SimpleKafkaTriggers.cs#L18

shrohilla avatar May 10 '22 17:05 shrohilla

I would love to see a full example if possible, because I'm not able to figure out a few things. Like, in that same you linked to, there is one function commented out and the other is not... how can I do it so that there is one function and it works in multiple environments? I need it to be one function that can work both locally and in azure with only environment variables differentiating it... and I can't seem to find a full example of this.

Like for example how am I supposed to configure the Protocol field on the [KafkaTrigger] attribute? I need the protocol to be set via environment variable but its an enum and so I can't do this:

[KafkaTrigger(Protocol = "%KafkaProtocol%")]

This doesn't work because its the wrong type, so how do I assign an enum field via environment variables? Its not clear how that should work.

Here's my function:

[Function("hello_kafka")]
public async Task HelloKafka(
  [KafkaTrigger(
    "%KafkaBootstrapServers%",
    "%KafkaTopicsHelloName%",
    ConsumerGroup = "%KafkaTopicsHelloConsumerGroup%",

    // Protocol = BrokerProtocol.NotSet,
    SslCaLocation = "%KafkaSslCaLocation%",
    SslCertificateLocation = "%KafkaSslCertificateLocation%",
    SslKeyLocation = "%KafkaSslKeyLocation%",
    SslKeyPassword = "%KafkaSslKeyPassword%",
    IsBatched = true
  )]
  string[] inputs)
{
    foreach (var message in KafkaMessage.Parse<GreetingMessage>(inputs))
    {
        Console.WriteLine($"Kafka Hello Input: {message.Value.Greeting}");
    }

    await Task.CompletedTask;
}

Its so ugly, it makes no sense why I have to set all these settings in every attribute instead of just configuring kafka globally.

It also works locally but fails when I try to connect to my actual Kafka server because I need to set the Protocol to be ssl, but I can't figure out how to do it without hard coding it right into this trigger attribute. Also, I can't configure these fields to use proper configuration sections instead I have to put them all in the "Values" field in the settings file, the whole thing is atrocious and I don't know how you can stand it other than to imagine that whoever is creating samples isn't actually trying a real-world scenario.

justinmchase avatar May 11 '22 03:05 justinmchase