Answer-is-not-42 comments

Repositories
Issues
Comments

Results 2 comments of


                                            Answer-is-not-42

Using llama.cpp, the entire context gets reprocessed each generation

I've managed to make cache work 3 times by typing in input, clicking "send dummy message" then "continue", then it stopped working. It's not consistent, can just decide to reprocess...

Using llama.cpp, the entire context gets reprocessed each generation

@eiery It's may partially be because of that, but the same thing(no cache hit) occurs in default interface mode too! Even without adding anything and just hitting "continue", it recomputes...