FastChat fastchat.model.apply

Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████| 41/41 [00:28<00:00, 1.44it/s] Loading delta Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████| 3/3 [00:18<00:00, 6.16s/it] Traceback (most recent call last): File "/data/anbo/anaconda3/envs/chat/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/data/anbo/anaconda3/envs/chat/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/data/anbo/github/FastChat-main/fastchat/model/apply_delta.py", line 49, in apply_delta(args.base_model_path, args.target_model_path, args.delta_path) File "/data/anbo/github/FastChat-main/fastchat/model/apply_delta.py", line 22, in apply_delta base_tokenizer = AutoTokenizer.from_pretrained(base_model_path) File "/data/anbo/anaconda3/envs/chat/lib/python3.8/site-packages/transformers-4.28.0.dev0-py3.8.egg/transformers/models/auto/tokenization_auto.py", line 682, in from_pretrained return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs) File "/data/anbo/anaconda3/envs/chat/lib/python3.8/site-packages/transformers-4.28.0.dev0-py3.8.egg/transformers/tokenization_utils_base.py", line 1805, in from_pretrained return cls._from_pretrained( File "/data/anbo/anaconda3/envs/chat/lib/python3.8/site-packages/transformers-4.28.0.dev0-py3.8.egg/transformers/tokenization_utils_base.py", line 1959, in _from_pretrained tokenizer = cls(*init_inputs, **init_kwargs) File "/data/anbo/anaconda3/envs/chat/lib/python3.8/site-packages/transformers-4.28.0.dev0-py3.8.egg/transformers/models/llama/tokenization_llama.py", line 71, in init self.sp_model.Load(vocab_file) File "/data/anbo/anaconda3/envs/chat/lib/python3.8/site-packages/sentencepiece/init.py", line 905, in Load return self.LoadFromFile(model_file) File "/data/anbo/anaconda3/envs/chat/lib/python3.8/site-packages/sentencepiece/init.py", line 310, in LoadFromFile return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg) TypeError: not a string

tokenizers==0.12.1 toolz==0.12.0 torch==2.0.0 tqdm==4.65.0 transformers==4.28.0.dev0

sentencepiece==0.1.97 sentry-sdk==1.17.0 setproctitle==1.3.2 setuptools-rust==1.5.2 six==1.16.0

Apr 07 '23 02:04 anbo724

Had a similar issue, did a couple things and was able to apply the deltas, I'm not sure which specific one worked.

Using conda on wsl:

conda create -n fastchat python=3.9
conda activate fastchat 
pip3 install fschat
pip install git+https://github.com/huggingface/transformers@cae78c46d
python3 -m fastchat.model.apply_delta --base /path/to/llama13b --target /path/to/vicuna13b --delta lmsys/vicuna-13b-delta-v0

Specific changes:

Using this version of hugging face transformers, instead of latest: transformers@cae78c46d
Switched from using a downloaded version of the deltas to the ones hosted on hugging face.

Apr 07 '23 19:04 AmpleSoros

Had a similar issue, did a couple things and was able to apply the deltas, I'm not sure which specific one worked.

Using conda on wsl:
conda create -n fastchat python=3.9
conda activate fastchat 
pip3 install fschat
pip install git+https://github.com/huggingface/transformers@cae78c46d
python3 -m fastchat.model.apply_delta --base /path/to/llama13b --target /path/to/vicuna13b --delta lmsys/vicuna-13b-delta-v0
Specific changes:

Using this version of hugging face transformers, instead of latest: transformers@cae78c46d

Switched from using a downloaded version of the deltas to the ones hosted on hugging face.

I followed the steps above, but still encounter the same error. My error occurs during the loading process of the 41st file, which is different from the one mentioned above.