continue
continue copied to clipboard
Support Cohere's Command-R models on open-source providers (e.g. Ollama)
Validations
- [X] I believe this is a way to improve. I'll try to join the Continue Discord for questions
- [X] I'm not able to find an open issue that requests the same enhancement
Problem
CohereForAI released the weights of the Command-R models for research on HuggingFace. It also means they can used (non-commercially) with providers like Ollama
Solution
- The models must be added to the
osModelslist: https://github.com/continuedev/continue/blob/d40b4c5d4bf593bc81f0b909ccae9837e8785bfb/gui/src/util/modelData.ts#L533 - Add a prompt template to core/llm/templates/chat.ts
- The usage of the
autodetectTemplateTypefunction might need to be refactored to take the provider in addition to the model for it to add the required template in this case (the Cohere API does not need one) https://github.com/continuedev/continue/blob/36a0800677887b33d2c36eeecf6c62c897152fdd/core/llm/autodetect.ts#L104