continue icon indicating copy to clipboard operation
continue copied to clipboard

Support Cohere's Command-R models on open-source providers (e.g. Ollama)

Open maxbrunet opened this issue 1 year ago • 0 comments

Validations

  • [X] I believe this is a way to improve. I'll try to join the Continue Discord for questions
  • [X] I'm not able to find an open issue that requests the same enhancement

Problem

CohereForAI released the weights of the Command-R models for research on HuggingFace. It also means they can used (non-commercially) with providers like Ollama

Solution

  1. The models must be added to the osModels list: https://github.com/continuedev/continue/blob/d40b4c5d4bf593bc81f0b909ccae9837e8785bfb/gui/src/util/modelData.ts#L533
  2. Add a prompt template to core/llm/templates/chat.ts
  3. The usage of the autodetectTemplateType function might need to be refactored to take the provider in addition to the model for it to add the required template in this case (the Cohere API does not need one) https://github.com/continuedev/continue/blob/36a0800677887b33d2c36eeecf6c62c897152fdd/core/llm/autodetect.ts#L104

maxbrunet avatar Apr 15 '24 05:04 maxbrunet