feature: allow runner signatures without arguments

[mqk]

Feature request

I would it if runner signatures could be called without an input_data argument.

Motivation

We want to be able to access the feature names from a trained LightGBM classifier. The model has a method called feature_name() on it (https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.feature_name). We tried to add a signature for it to the runner, so that we could access it in the bento service, but that failed because runner signatures require an argument.

Other

No response

[ssheng]

Thanks for reporting this issue, @mqk . I don’t think there is a strong requirement for runner method to have at least one argument. cc: @bojiang

[parano]

Thanks for the feedback @mqk!

Another approach is to save the feature names in the model's custom_objects field, here's an example: https://github.com/bentoml/BentoML/tree/main/examples/sklearn/pipeline

...

bento_model = bentoml.sklearn.save_model(
    "20_news_group",
    grid_search.best_estimator_,
    signatures={
        "predict": {"batchable": True, "batch_dim": 0},
        "predict_proba": {"batchable": True, "batch_dim": 0},
    },
    custom_objects={
        "target_names": data.target_names,
    },
    metadata=best_parameters,
)
print(f"Model saved: {bento_model}")

bento_model = bentoml.sklearn.get("20_news_group:latest")
model_runner = bento_model.to_runner()
target_names = bento_model.custom_objects["target_names"]

svc = bentoml.Service("doc_classifier", runners=[model_runner])


@svc.api(input=Text(), output=JSON())
def predict(input_doc: str):
    prediction = model_runner.predict.run([input_doc])[0]
    return {"result": target_names[prediction]}

[mqk]

Thanks both. custom_objects is an interesting suggestion, @parano. Thanks, I'll investigate that solution. Still think it might be useful to allow users to expose methods on the model that don't require input.

[sauyon]

It's worth noting that the runner architecture is really designed for processes that are compute-heavy, and in most cases like this one it's pretty likely that using a runner will add unnecessary overhead to your service. Precomputing the values and storing it in the custom objects should be much more efficient.

While I do think we might want to support this eventually for some use case (random number generation, maybe?), it won't be a priority for the time being. As a side note we'd probably want to document reasonably well that use of constant methods like this on runners is discouraged.

[mqk]

Thanks, that makes sense, @sauyon. I'm fine with closing this issue.

[sauyon]

I think we're ok with leaving this one open since we do intend to do this eventually, it definitely makes sense to have eventually! Just wanted to let you know that we wouldn't be prioritizing it and why.

/cc @parano just in case