EventHub output binding: set partitionKey for output event

[tartieret]

Context

The EventHub SDK allows to specify the partitionKey for a batch of events, as shown here:

batch = hub_client.create_batch(partition_key=mykey)
batch.add(EventData(json.dumps({ ... })))
hub_client.send_batch(batch)

Issue

The documentation for Azure Functions shows how to use a function return value to send an event to EventHub, using the following configuration and function:

{
    "type": "eventHub",
    "name": "$return",
    "eventHubName": "myeventhub",
    "connection": "MyEventHubSendAppSetting",
    "direction": "out"
}

import logging
import azure.functions as func

def main(timer: func.TimerRequest) -> str:
    return 'Hello world'

How can we modify the example above to set the partitionKey on the output message ?

[anthonychu]

Hey @tartieret. This is not possible today. Are you able to bring in the Event Hubs SDK instead of using the binding?

[tartieret]

Hi @anthonychu I tried using the Event Hub SDK directly in an Azure Function, but the issue is that the function has to initiate a new connection at each execution, and this is quite slow, so the function regularly times out.

The challenge is that the data has to be partitionned a certain way to allow a "parallel" query in Stream Analytics (per "sensor" in my IOT scenario). However, my system is receiving data "per device" (where we have several sensors connected to the same device/gateway), so I have to re-partition the data. I was hoping to do that with an Azure Function using event hub input and output bindings.

[jbevarts]

Perhaps this is the wrong thread, but the context seems relevant.

In the example shown, @tartieret uses batch = client.creatBatch(). I'd like to enhance some code to remove dependency on the client and simply use output bindings. How can I take advantage of batch processing without using the eh client?

[anthonychu]

@jbevarts Looks like the output binding should take a list. @Hazhzeng Can you confirm the behavior? Perhaps docs need updating, as the JavaScript example shows how to return a batch.

@tartieret Sorry for the delay. You might be able to create the Event Hubs client once and store it in a global variable, reusing it across different function invocations.

Looks like in order to supporting outputting a partition key, we'd have to make significant modifications to the Event Hubs extension. @Hazhzeng do you know if this is true?

[tartieret]

Thanks @anthonychu We found a work-around without using an Azure Function, so it's not critical for me anymore.

[barbara-desousa-kroger]

Any update on this?

[cgreco-cm]

I'd also be interested in utilizing this functionality!