Overview

Azure Functions scales event hubs based on the number of unprocessed events distributed across all the partitions in the hub. By default it takes the configured maximum batch size and assumes that each allocated instance will process that many events, and allocates sufficient instances to process all unprocessed events, taking into account that each partition can only be processed by one instance. You can adjust this default behavior by specifying the TargetUnprocessedEventThreshold - the total unprocessed event count will be divided by this value to determine the number of instances, and then we will round that up to an instance count that creates a balanced partition distribution.

This PR introduces the new attribute that users can set to fine-tune the default target based scaling behavior, and it also adds support for target based scaling for Event Hub-triggered functions.

Checklist for Target Based Scaling Changes

[x] create EventHubsMetricsProvider
[x] remove duplicate logic between EventHubsSaleMonitor, EventHubsMetricsProvider
[x] EventHubsListener implementITargetScalerProvider
[x] fix unit tests
[x] add EventHubsargetScaler
[x] add unit tests for EventHubsTargetScaler
[x] E2E tests with Event Hub trigger with locally running function app
[ ] test with scale controller prototype (dependent on finalizing the scalerfactory implementation)

Logs from E2E Verification With Local Function App

Steps:

Generate a local nuget package with EventHub changes from this PR
Create function app referencing local EventHub nuget packages
Publish and run the application locally with azure-functions-host repo
Enable runtime driven scaling, target based scaling, and add ScmHostingConfigurations.txt to enable EventHubs TBS
Send 200k messages to event hub with 5 partitions, in batches of 300 messages
Ping admin/host/scale/status API using Postman, observe target worker count and local azure-functions-host logs emitted

Target Based Scaling Enabled