mindcraft
mindcraft copied to clipboard
Llama3 model takes super long time to load.
Whenever I give the llama3 model a prompt it takes about 5 minutes to actually respond and sometimes is just times out. When I use a smaller model like Tinyllama it goes faster but tinyllama has no idea how to use the mindcraft tools. My computer is powerful enough to run the model pretty fast and llama3 can answer a prompt in a couple seconds on my computer, but when I use the mindcraft thing it is super laggy. My laptop specs are
Processor : AMD Ryzen™ 9 6900HS with Radeon™ Graphics × 16
Memory : 16.0 GiB
Graphics : AMD Radeon™ Graphics / AMD Radeon™ RX 6700S
And I am running Debian 12.