Qiushan comments

Repositories
Issues
Comments

Results 2 comments of


                                            Qiushan

[Usage] tokenization mismatch when finetuning v1.5-7b

> @haotian-liu In my experiment, tokenizer set "use_fast=True" works , with transformers==4.34.1 and tokenizers==0.14.1. But don't know why mismatch when set "use_fast=False" @zzzzzzrc I tried to set "use_fast=True" and it...

[Usage] tokenization mismatch when finetuning v1.5-7b

> Is here fixed? Setting use_fast=True works for my case.