azure-event-hubs-spark icon indicating copy to clipboard operation
azure-event-hubs-spark copied to clipboard

Published 20 hours ago • verified publisher icon Azure

Allow multiple partition senders to be open at once

Open sabeegrewal opened this issue 6 years ago • 2 comments

Currently, if another partition sender is needed, the existing one is closed.

sabeegrewal avatar Jun 04 '18 21:06 sabeegrewal

Is this the issue that's causing preventing Databricks from opening up more than one EventHub sinks on a single spark cluster?

luzlab avatar Sep 11 '18 01:09 luzlab

No, this doesn't prevent anything in the sink - the current implementation is slow if your schema has partition in it. If you're sending round-robin or with a partition key, then this isn't relevant (and doesn't affect performance).

sabeegrewal avatar Sep 11 '18 01:09 sabeegrewal