Shuo Yang

Its tokenizer has a different name `LlamaTokenizer` @RedmiS22018

I agree. We just need to support the normal weight

I am also trying to deploy vicuna locally on auto-gpt, and I will try running your code to see where the issue lies. @sablin39

@sablin39 This issue might be caused by not changing the model name, as autogpt defaults to using gpt3.5-turbo. Can you successfully run it after changing the model name to vicuna-7b-v1.1?

Can you provide your `.env` file and your running command? I want to reproduce the problem @sablin39

please change your `.env` file with: ~~~ FAST_TOKEN_LIMIT=2560 ~~~ GPT-3.5 max token size is not equal to vicuna which caused this problem. However, directly connecting vicuna api with autogpt might...

Nice work! @jstzwj I have taken the time to run the code, and overall, I believe it functions well without any major issues. I would like to suggest some improvements...

I am working on reducing the peak memory, and the peak memory now is 11372.2 MiB. See #402 and we can further improve it.

I encountered the exact same problem; my code works well on an AWS g5 instance, but it doesn't work on an AWS g4 large instance. I compiled the code: ~~~c...

After launching the server: ~~~bash Using a slow tokenizer. This might cause a significant slowdown. Consider using a fast tokenizer instead. Using a slow tokenizer. This might cause a significant...