Invoking a non-HTTP triggered function locally using the Azure Functions Core Tools Admin API (cardinality: many)

[amotl]

trafficstars

Is your question related to a specific version? If so, please specify:

Found Python version 3.8.5 (python3). Azure Functions Core Tools (3.0.2931 Commit hash: d552c6741a37422684f0efab41d541ebad2b2bd2) Function Runtime Version: 3.0.14492.0

What binding does your question apply to, if any? (e.g. Blob Trigger, Event Hub Binding, etc)

Event Hub Trigger Binding (cardinality: many)

Question

Hi there,

we are trying to follow the documentation about how to pass test data to a non-HTTP triggered function to invoke a Python function locally when running within the Azure Functions Core Tools host.

Within #764, @kepalas has been able to figure out how to invoke the function successfully when bound with cardinality: one - thank you so much! Within this issue, we would like to focus on cardinality: many instead.

With kind regards, Andreas.

[amotl]

Hi again,

as a followup to #764, we are trying to figure out how to submit a list of events, reflected by a binding attribute "cardinality": "many" within function.json like that:

http http://0.0.0.0:7071/admin/functions/eventHubTrigger \
  input='{"SystemPropertiesArray":[{}],"data":[{"foo":"Ping from Az CLI IoT Extension #1","bar":{"id":"810d9efd-458c-49f2-a9c0-822c897a02e3","timestamp":"2020-10-28 23:24:16.315294"}}]}' \
  --print hHbB

The SystemPropertiesArray idea has been picked up from inspecting the source code of EventHubTriggerConverter.decode_multiple_events(), EventHubTriggerConverter._is_cardinality_many() and some of the tests within azure-functions-python-worker.

The idea to supply a single empty dictionary to SystemPropertiesArray has also been taken from the len(parsed_data) == len(parsed_sys_props) assertion within EventHubTriggerConverter.decode_multiple_events().

However, I have no idea how to wrap the actual list of events. Using data here does not work yet.

According to EventHubConverter.decode(), one would have to somehow submit data of type collection_bytes or collection_string instead of string, bytes or json. Together with the grpc definition message TypedData, that might indicate that it is not possible to invoke a function binding to cardinality: many using plain JSON at all and instead it would only be possible using Protocol Buffers?

As far as we can see, one would have to supply the right input format to let the payload be funneled through EventHubTriggerConverter.decode_multiple_events() somehow, right?

While https://github.com/Azure/azure-functions-host/issues/3370 is not exactly about invoking the function locally, it shows at least that invoking a function which binds to some sort of collection (aka. cardinality: many) might not be as easy as for functions which bind to single events (aka. cardinality: one).

Maybe you are able to apply some additional wizardry from your pen, @kepalas?

With kind regards, Andreas.

[amotl]

one would have to somehow submit data of type collection_bytes or collection_string instead of string, bytes or json

According to EventHubTriggerConverter.decode_multiple_events(), it should be possible to submit data of type json. However, data seems not to make it through to this method.

For investigating this further, we added the snippet

with open("/tmp/hello", "w") as f:
    f.write(str(trigger_metadata.keys()))

to the beginning of the EventHubTriggerConverter.decode() method. The reason behind that is that we haven't been able to print() anything sensible to neither stdout nor stderr.

The outcome for "cardinality: one" is:

dict_keys(['foo', 'Properties', 'SystemProperties', 'events', 'EnqueuedTimeUtc', 'bar', 'SequenceNumber'])

The outcome for "cardinality: many" is:

dict_keys(['Properties', 'SequenceNumber', 'events', 'EnqueuedTimeUtc'])

The consequence of this is that _is_cardinality_one() will work and return True, as SystemProperties is contained within the dictionary. However, in the case of cardinality: many, it will not work because _is_cardinality_many() will return False as SystemPropertiesArray is not contained within the dictionary.

I hope this helps for further getting more ideas what might be going on here. Maybe @anthonychu, @Hazhzeng or @vrdmr are also able to add some thoughts to this?

[imshinto]

Hi @amotl , Were you able to crack this issue ? We are also trying to send a batch of events using the admin api (and keys). But unfortunately the structure we receive in functions is not what we are expecting.