llama.cpp icon indicating copy to clipboard operation
llama.cpp copied to clipboard

llama : refactor get / set state + remove redundant kv cache API

Open ggerganov opened this issue 1 year ago • 0 comments

  • Normalize the code style
  • Move the definitions at the correct place in llama.cpp
  • Retire llama_get_kv_cache(), llama_get_kv_cache_size() and llama_set_kv_cache()

Not sure how to test this - maybe we need to add an example, or extend main with store/load state functionality

ggerganov avatar Apr 23 '23 15:04 ggerganov