langchaingo icon indicating copy to clipboard operation
langchaingo copied to clipboard
verified publisher icon tmc

CallOptions for both langchaingo's and provider side's cache control

Open leventov opened this issue 1 year ago • 6 comments

Langchaingo's side:

add options like CacherSkipGet bool and CacherSkipPut bool that are understood only by Cacher as llms.Model impl, with clear semantics.

Provider's side:

Needed to support Anthropic prompt caching.

leventov avatar Dec 13 '24 06:12 leventov

Is it all model have?

lifejwang11 avatar Jan 07 '25 13:01 lifejwang11

@lifejwang11 I don't understand your question

leventov avatar Jan 09 '25 08:01 leventov

@leventov Is this feature unique to this model or does it exist in all other models

lifejwang11 avatar Jan 09 '25 08:01 lifejwang11

I don't know.

The Langchaingo's side cache control (which I'm actually using; I've already implemented this in my fork of langchaingo, but didn't publish a PR yet) would be agnostic of the model, or course.

leventov avatar Jan 09 '25 12:01 leventov

you can push the pr to main,I think everyone need this feature

lifejwang11 avatar Jan 10 '25 01:01 lifejwang11