feature: custom objects/models accepting serialization functions instead of relying on cloudpickle

[cadmusthefounder]

Feature request

In the docs, it is mentioned that for custom objects like tokenizer, the object is being pickled

bentoml.pytorch.save_model(
    "demo_mnist",   # model name in the local model store
    trained_model,  # model instance being saved
    labels={    # user-defined labels for managing models in Yatai
        "owner": "nlp_team",
        "stage": "dev",
    },
    metadata={  # user-defined additional metadata
        "acc": acc,
        "cv_stats": cv_stats,
        "dataset_version": "20210820",
    },
    custom_objects={    # save additional user-defined python objects
        "tokenizer": tokenizer_object,
    }
)

and it is similar for custom models. This may not be desirable due to the explanation in the next section. Suggestion would be to allow the saving of other file types in addition to the model file, as well as allowing users to define their own functions to convert those files into python objects.

Motivation

Main problem with pickle is that things may break when the python version changes. To maintain backwards compatibility, users should be able to define the serialization and deserialization logic, as well as handling other file types.

Other

No response

[SebastianScherer88]

related suggestion: extend the way those extra files appear in the resulting model store entry to something more along the lines of the bento builds' include argument. i.e. instead of having an intransparent custom_objects.pkl pickle file as part of the exported model, it would be nice to have:

• the ability to pass files, not just python objects
• the resulting object files saved into a flat directory structure, much like what is already happening at the bento build phase.

this fits in with the original request in that the huggingface tokenizers's native save method exports a collection of files to a specified directory. the tokenizer's from_pretrained method can then be pointed at the same directory, reading in whats needed and discarding non-tokenizer related files that are also placed in that directory.

my team cant use the Transformer framework as we require our models to be in the neuron torchscript format, which the huggingface pipeline currently doesnt support afaik.

even without the neuron torchscript motivation this would be a nice extension of the current model export functionality, making the storage of the pre-/postprocessing artifacts in the bento model store a bit more transparent imo :)

[frostming]

We recommend saving these objects as separate bentoml models, and load them together into the service when it starts up.