MLServer
MLServer copied to clipboard
Issue with models that needs trust_remote_code
Some models in HuggingFace hub needs trust_remote_code to be put at True in order to run these models.
For example trying to run this model with MlServer https://huggingface.co/tiiuae/falcon-7b-instruct we obtain
ValueError: Loading tiiuae/falcon-7b-instruct requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option `trust_remote_code=True` to remove this error.
I'm not seeing a way to enable this flag using MlServer, maybe should be exported?
Thank you :)