Bin Lei

Deepseeker model completely loses performance after using tokenizer.add_tokens(special_tokens)

1

Hey guys, I am trying to add some customized new tokens in a Code related model `deepseek-ai/deepseek-coder-6.7b-instruct`. However the model **completely loses performance** after using `tokenizer.add_tokens(special_tokens)`. Here is how I...

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction 求大佬们帮忙看下这个问题: **问题描述:** 在slurm上多机多卡进行sft处理数据集时只要preprocessing_num_workers > 1, 程序就会卡在 `/LLaMA-Factory/src/llmtuner/train/sft/workflow.py`的这一步: `dataset = get_dataset(tokenizer, model_args, data_args, training_args, stage="sft")`....

### Model introduction The model outperforms the GPT-4 Turbo on the HumanEval base test **(90.9% vs 90.2%)**. And it provides a new kind of code Interpreter. ### Model URL https://huggingface.co/Bin12345/AutoCoder...

Hello DeepSeek Team, thanks for your great work! I fine-tuned your previous DeepSeek-Coder 33B model and got a model which performers well on the HumanEval benchmark. https://github.com/bin123apple/AutoCoder. However, while testing...

Some cases seem correct, but the results are 0.0.

5

Dear Authors, thanks for the great work! I am testing my agent, some cases seems correctly finished, however, the final result is `0`. I am not sure if there is...