How to set trust_remote_code=True in pipeline

[gujingit]

Hi, how to set trust_remote_code=True in mii pipeline?

Code

from mii import pipeline
pipe = pipeline("/root/Qwen-7B-Chat")
pipe.tokenizer.tokenizer.eos_token_id = 151643
output = pipe(["DeepSpeed is"], max_new_tokens=128, do_sample=False)
print(output)

Output

The repository for /root/Qwen-7B-Chat contains custom code which must be executed to correctly load the model. You can inspect the repository content at https://hf.co//root/Qwen-7B-Chat.
You can avoid this prompt in future by passing the argument `trust_remote_code=True`.

Do you wish to run the custom code? [y/N]  

[ZonePG]

I think that there isn't currently an exposed interface for the 'trust_remote_code' parameter. You can set it by modifying the load method directly in the deepspeed source code.

https://github.com/microsoft/DeepSpeed/blob/2a6c58df34ea04e0aa451a4623b7c7d49b087a50/deepspeed/inference/v2/checkpoint/huggingface_engine.py#L24

[mrwyattii]

Hi @gujingit unfortunately @ZonePG is correct. I think this addition is long overdue. Let me put together a PR that adds a hf_kwargs option to our API. This should allow for arbitrary args to be passed for loading the checkpoint weights from HuggingFace.