MLServer
MLServer copied to clipboard
Consider adding a utility for retrieving inputs by name
Inputs are a list of tensors with a name entry, however it's not possible to use this so select tensors by name. Instead one must either resort to index-based selection which can be error-prone or to do a pass on the whole list of inputs and build a dictionary by hand:
inputs = {}
for input in payload.inputs:
inputs[input.name] = self.decode(input)
Adding this functionality would also make it easier to compose input requests since the order of the inputs becomes less meaningful if we know the server handles the inputs by name (although this perhaps introduces a pattern going against the philosophy of V2 where inputs should be in a canonical order?).
There is precedent for this in the Triton Python backend: https://github.com/triton-inference-server/python_backend/blob/41d646011653af6d5f9955b523fbfc121ab4dd54/src/resources/triton_python_backend_utils.py#L122-L142