aiXcoder-7B icon indicating copy to clipboard operation
aiXcoder-7B copied to clipboard

[New Feature] Add some instructions for Hugging Face based methods

Open YoctoHan opened this issue 1 year ago • 2 comments

While previous version didn't explain how the weights should be loaded, most runs used the default loading method (downloaded from the modelscope), this version some instructions will be added to help those who want to run the model based on the hugging face framework loading local weight.

YoctoHan avatar Apr 18 '24 06:04 YoctoHan

i can help if you are reaching otu to outside collaborators for this :-)

Josephrp avatar Apr 19 '24 05:04 Josephrp

Thank you for your kind assistance.

If you have downloaded model weights from HuggingFace or ModelScope through various methods, you can change the inference code to your local address for inference. For example, if I download the model to the local directory "D:\data\aix-7b-model", then the inference code can be modified to load the model from this location:


tokenizer = AutoTokenizer.from_pretrained("D:\data\aix-7b-model")
model = AutoModelForCausalLM.from_pretrained("D:\data\aix-7b-model", torch_dtype=torch.bfloat16)

HoratioJSY avatar Apr 19 '24 07:04 HoratioJSY