djl-serving
djl-serving copied to clipboard
[AOT] Tokenizer should be optional for AOT
Description
Right now, in our default handlers, tokenizer is expected.
Right behaviour:
- If HF model_id is provided, then we get the tokenizer from there and just download it for them to make it ready for inference. [existing flow should cover this]
- If they use their own files, if tokenizer is available then we copy it, otherwise we should not expect it. We could get the tokenizer file path from the command line argument and copy it for them as well.
cc @frankfliu