Support array of parameters

[jamesanto]

Description

Supports array of parameters. It might be useful in cases where the model might expect a list of images, instead of a single image. It could also be used to control batching manually.

Fixes #1536 (Potentially/Partially ?)

Type of change

• [x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [x] This change requires a documentation update

Technically it doesn't break any API/interface. The service would behave differently if someone uses array parameters in requests.

Feature/Issue validation/testing

Added unit test validating the expected request formats.

Checklist:

• [x] Did you have fun?
• [x] Have you added tests that prove your fix is effective or that this feature works?
• [ ] Has code been commented, particularly in hard-to-understand areas?
• [ ] Have you made corresponding changes to the documentation?

[msaroufim]

So I may need some more time to review and think about this one. Overall I think this makes sense to support video and time series use cases. As far as controlling the batch size though this probably isn't the right use case since we already support dynamic batching. Would you be interested in working on a video or time series example to make the case for this change?

[lxning]

There are two scenarios:

1. the input data of a model is an array.
2. the input is a batch of inference requests.

Solutions: scenario 1) set batch size =1 and input multiple data scenario 2) set batch size = N and input multiple data

• how to input multiple data curl multiple requests or How can pass multiple images in Inference request call to my model?

[msaroufim]

I believe per @lxning's comment this PR is not needed but feel free to reopen if you'd like to revisit