JaonLiu
JaonLiu
> > where the heck do you install models??? where to put them???? NO DOC or whatsoever....... > > Choose one of them.  > > Then run it, it...
4张A100(40G)，batch size=1，对Baichuan2-7B-Base进行全量微调，都OOV
I install flash-attention by `pip install flash-attn --no-build-isolation` ,How can I install `rotary_emb`? otherwise comes error: ``` import rotary_emb ModuleNotFoundError: No module named 'rotary_emb' ```
> `cd csrc/rotary && python setup.py install` @tridao That means even though I install flash-attn by pip, i have to download source code so that I can run `cd csrc/rotary...
> pip install "git+https://github.com/Dao-AILab/flash-attention.git#subdirectory=csrc/rotary Thanks a lot, `pip install "git+https://github.com/Dao-AILab/flash-attention.git#subdirectory=csrc/rotary"` work for me
> 回译一句一行，末尾\n youdao_translate.py, line 72。里面有个`f.write(t[0] + '\t' + t[1].strip() + '\n')`。所以，按照您的数据组织方式，是会运行出错的。
> @CerrieJ @linchen111 @Zhutianyi7230 - hi all > > ``` > File "/home/l00841998/DeepSpeedExamples/applications/DeepSpeed-Chat/training/utils/data/raw_datasets.py", line 20, in __init__ > self.raw_datasets = load_from_disk(dataset_name) > ``` > > change the `load_from_disk` to `load_dataset`...
> @MrRace - the arg `dataset_name` should be like `/media/datasets/full-hh-rlhf/data`. Thanks a lot!