[feature request] Add a model versionning

[ThomasDelteil]

When building models it would be nice to have a field to add a version of the model in the signature.json file, this model version would be available in the api-description end-point.

[lupesko]

Good suggestion. We can probably generalize and add more things like a free-text description field, org name, license and more. Basically many things that goes into a JAR or .NET DLL. I'm not sure whether signature.json is the right place though.

[vdantu]

@ThomasDelteil : Could you describe this requirement a little more? Do you intend to use this version to do some sort of A/B testing? Or, do you see any value in having different endpoints for different versions of the same model? I would love to learn more about the reasoning behind this issue.

[ThomasDelteil]

The idea is to let the consumer of the API know what version of the model it is consuming, to help with debugging for example. Being able to hit the "api/description" end-point and access that information helps understand which model is running.

For example let's say I am rolling a new version of my model, and want to make sure that the requests are routed to the new container rather than the old and that the end-to-end routing is working, right now I wouldn't have a way to do that easily directly from the client.

Another use-case is for example, let's say I am building an app that heavily consumes a MMS backed API, for debugging purposes I would probably call to "api/description" and log which models are available and which version they are running.

Another use-case is let say you are experiencing strange predictions of your model, by the time you are actually investigating, you might have automatically updated your models a few times. If in the MMS logs you also print the model version tags, alongside the timestamp, it makes it a lot simpler to know which model was faulty and track the root cause issue, rather than trying to correlate with other logs etc.

[vdantu]

@ThomasDelteil : Currently we do have a capacity to add version numbers to MANIFEST, but its manual process at the moment. Meaning, we will have to give a version number when we are creating the model archive file. We could display it in the stats, but we also let customers create custom-endpoints per model. Wouldn't that be enough? Customers could create

mxnet-model-server --models customModelV1=https://s3...customModel.mar customModelV2=https://s3.aws...customModel2.mar

Now customers can pull the endpoints available through a REST call , eg

curl localhost:8081/models

(if you are running model server on localhost with default management port). This could tell customers, which containers are running which version of the model.

We could add the version info in our "details show as well", but I would think its simpler and easier to get this information from the endpoint.

For the final requirement, you make a good point. I think we could add the model-name to the logs. This will definitely make it easier to debug the models. We will look into adding it.