Paweł Zygler

I have same issue. Noone of those chunk size or overlap changes made difference. pt_tokenize: unknown token 'n' gpt_tokenize: unknown token 's' gpt_tokenize: unknown token 'w' gpt_tokenize: unknown token 'e'...

Anyone found a solution?