lmql icon indicating copy to clipboard operation
lmql copied to clipboard

Published 20 hours ago • verified publisher icon eth-sri

Support Anthropic's Claude LLM

Open aretrace opened this issue 2 years ago • 1 comments

Having a Claude option would be nice.

aretrace avatar Jul 17 '23 02:07 aretrace

We intend to add more model backends over time. For Anthropic models, it looks like the API is still very restricted: https://docs.anthropic.com/claude/reference/complete_post.

More specifically, it does not offer a way to mask the model distribution via a logit bias. This means we can add support, but full constraining/model steering will not be possible, due to API restrictions on their end.

Still, I'll keep this issue around, adding support is definitely a good suggestion.

lbeurerkellner avatar Jul 17 '23 08:07 lbeurerkellner