How to use the model that has already been downloaded？

Open AntonioZC666 opened this issue 2 years ago • 2 comments

Hello, I'm trying to use FlexGen. I would like to know if I can run the project with a model that I have already downloaded? I'm using the case command line python3 -m flexgen.flex_opt --model facebook/opt-1.3b and it automatically downloads the model from Huggingface, but I've already downloaded it myself in advance. I tried using other parameter such as '--path' and adding the file path where my model is located, but it didn't work.

I just want to use a model that I've already downloaded, how should I do that?

Jan 07 '24 05:01 AntonioZC666

And I searched for the relevant Q&A and I found an argument --local from PR #111 I used a command python3 -m flexgen.flex_opt --model /mnt/data/share/llm/models/hf/opt-1.3b --local, but it didn't work. The error is flex_opt.py: error: unrecognized arguments: --local. How can I make it?

Jan 10 '24 01:01 AntonioZC666