mindcraft icon indicating copy to clipboard operation
mindcraft copied to clipboard

Llama3 model takes super long time to load.

Open Ligerbot opened this issue 6 months ago • 4 comments

Whenever I give the llama3 model a prompt it takes about 5 minutes to actually respond and sometimes is just times out. When I use a smaller model like Tinyllama it goes faster but tinyllama has no idea how to use the mindcraft tools. My computer is powerful enough to run the model pretty fast and llama3 can answer a prompt in a couple seconds on my computer, but when I use the mindcraft thing it is super laggy. My laptop specs are

Processor : AMD Ryzen™ 9 6900HS with Radeon™ Graphics × 16
Memory : 16.0 GiB
Graphics : AMD Radeon™ Graphics / AMD Radeon™ RX 6700S

And I am running Debian 12.

Ligerbot avatar Aug 25 '24 03:08 Ligerbot