MLServer icon indicating copy to clipboard operation
MLServer copied to clipboard

Issue with models that needs trust_remote_code

Open rivamarco opened this issue 2 years ago • 0 comments

Some models in HuggingFace hub needs trust_remote_code to be put at True in order to run these models.

For example trying to run this model with MlServer https://huggingface.co/tiiuae/falcon-7b-instruct we obtain

ValueError: Loading tiiuae/falcon-7b-instruct requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option `trust_remote_code=True` to remove this error.

I'm not seeing a way to enable this flag using MlServer, maybe should be exported?

Thank you :)

rivamarco avatar Jul 20 '23 08:07 rivamarco