GithubX-F

same question. Are there any examples provided?

> 一个快速修复方法是使用以下命令加载标记器： > > ```python > from transformers import RobertaTokenizerFast > > tokenizer = RobertaTokenizerFast.from_pretrained("answerdotai/ModernBERT-large", add_prefix_space=True) > ``` > > 通过我的 Flair 基准测试，我现在能够在开发集上获得 96.24%，在测试集上获得 92.01%。我目前正在使用不同的超参数进行更多运行 :) Compared to the original...

Similarly, I encountered a similar issue where my loss was always 0, but when I replaced it with the BERT model, the loss decreased normally.

You can try FlashAttention2, for example: ```python AutoModelForMaskedLM.from_pretrained( MODEL_PATH, trust_remote_code=True, ignore_mismatched_sizes=True, torch_dtype=torch.bfloat16 if is_torch_bf16_gpu_available() else torch.float16, attn_implementation=ATTN_IMPLEMENTATION ).to("cuda" if torch.cuda.is_available() else "cpu") ```

> ``` > pip install modelscope==1.29.0 > ``` > > 我把模型范围仪版本回滚到 1.29.0，现在可以用了。希望这能帮到你。 This works. Good luck!