Nikolaos Papadopoulos

> This only occurs with some datasets, I suspect this maybe a bug. hey mind giving me an example of a dataset that works normally with the settings in your...

> My understanding is that ChatGPT runs `GPT 3.5` (whatever that means), and the published OpenAI endpoints use `GPT 3`. > > I'm not 100% sure if that means ChatGPT...

1. Training code has to change 2. Data that you fine tune the model with after training has to change So no, nothing can be done user-side to change attention...

 So 4096 tokens.

> Any experience in using it for more than 4096 tokens? any idea when checkpoints trained on more than 1 trillion tokens will be ready? If you have a conversation...

Getting the same error, gpt_tokenize: unknown token 'Γ' gpt_tokenize: unknown token 'Ç' gpt_tokenize: unknown token 'Ö' gpt_tokenize: unknown token 'Γ' gpt_tokenize: unknown token 'Ç' gpt_tokenize: unknown token 'Ö' gpt_tokenize: unknown...

Did you also do 3.Download the MinGW installer from the [MinGW website](https://sourceforge.net/projects/mingw/). 4.Run the installer and select the gcc component. Steps number 3,4 ? Restart afterwards,yes?

> This fixed it for me. > > Incorrect or incomplete installation of MSVC: Ensure you have MSVC properly installed with the correct toolset. In Visual Studio Installer, there is...

> > @JeongChangsu thank you for the suggestion! > > I tried to comment on that line but it gave me some strange warnings. But your suggestion helped me find...

I may be experiencing the same problem because of this package but it was obfuscated inside the httpx package that I recently started using (python 3.12) :(. My local deployments...