MLServer icon indicating copy to clipboard operation
MLServer copied to clipboard

Consider adding a utility for retrieving inputs by name

Open jklaise opened this issue 3 years ago • 0 comments
trafficstars

Inputs are a list of tensors with a name entry, however it's not possible to use this so select tensors by name. Instead one must either resort to index-based selection which can be error-prone or to do a pass on the whole list of inputs and build a dictionary by hand:

inputs = {}
for input in payload.inputs:
   inputs[input.name] = self.decode(input)

Adding this functionality would also make it easier to compose input requests since the order of the inputs becomes less meaningful if we know the server handles the inputs by name (although this perhaps introduces a pattern going against the philosophy of V2 where inputs should be in a canonical order?).

There is precedent for this in the Triton Python backend: https://github.com/triton-inference-server/python_backend/blob/41d646011653af6d5f9955b523fbfc121ab4dd54/src/resources/triton_python_backend_utils.py#L122-L142

jklaise avatar Apr 14 '22 10:04 jklaise